223 citations · 321 across the 16 of their papers we have counts for
10 papers · 1 filter
Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models
Yonggan Fu, Xin Dong, Shizhe Diao +12
Efficient deployment of small language models (SLMs) is essential for numerous real-world applications with stringent latency constraints. While previous work on SLM design has pri…
Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition
Lucas Liebenwein, Alaa Maalouf, Oren Gal +2
We present a novel global compression framework for deep neural networks that automatically analyzes each layer to identify the optimal per-layer compression ratio, while simultane…
Closed-form Continuous-time Neural Models
Ramin Hasani, Mathias Lechner, Alexander Amini +5
Continuous-time neural processes are performant sequential decision-makers that are built by differential equations (DE). However, their expressive power when they are deployed on…
Sparse Flows: Pruning Continuous-depth Models
Lucas Liebenwein, Ramin Hasani, Alexander Amini +1
Continuous deep learning architectures enable learning of flexible probabilistic models for predictive modeling as neural ordinary differential equations (ODEs), and for generative…
Low-Regret Active learning
Cenk Baykal, Lucas Liebenwein, Dan Feldman +1
We develop an online learning algorithm for identifying unlabeled data points that are most informative for training (i.e., active learning). By formulating the active learning pro…
Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy
Lucas Liebenwein, Cenk Baykal, Brandon Carter +2
Neural network pruning is a popular technique used to reduce the inference costs of modern, potentially overparameterized, networks. Starting from a pre-trained network, the proces…
Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space
Wilko Schwarting, Tim Seyde, Igor Gilitschenski +4
Learning competitive behaviors in multi-agent settings such as racing requires long-term reasoning about potential adversarial interactions. This paper presents Deep Latent Competi…
Provable Filter Pruning for Efficient Neural Networks
Lucas Liebenwein, Cenk Baykal, Harry Lang +2
We present a provable, sampling-based approach for generating compact Convolutional Neural Networks (CNNs) by identifying and removing redundant filters from an over-parameterized…
SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks
Cenk Baykal, Lucas Liebenwein, Igor Gilitschenski +2
We introduce a pruning algorithm that provably sparsifies the parameters of a trained model in a way that approximately preserves the model's predictive accuracy. Our algorithm use…
Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Bounds
Cenk Baykal, Lucas Liebenwein, Igor Gilitschenski +2
We present an efficient coresets-based neural network compression algorithm that sparsifies the parameters of a trained fully-connected neural network in a manner that provably app…