223 citations · 475 across the 51 of their papers we have counts for
31 papers · 1 filter
Riccati State Space Models: Non-iterative Parallelization for Nonlinear Sequence Modeling
Mónika Farsang, Ramin Hasani, Daniela Rus +1
State space models (SSMs) achieve efficient sequence processing because their affine state updates are closed under composition and can therefore be evaluated with an associative p…
Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence
Mónika Farsang, Ramin Hasani, Daniela Rus +1
State Space Models (SSMs) are increasingly deployed in the Edge because they offer, at comparable performance, a smaller memory/training/inference footprint, compared to Large Lang…
MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery
Maksim Kuznetsov, Zulfat Miftahutdinov, Rim Shayakhmetov +17
General-purpose large language models (LLMs) that rely on in-context learning do not reliably deliver the scientific understanding and performance required for drug discovery tasks…
LFM2 Technical Report
Alexander Amini, Anna Banaszak, Harold Benoit +30
We present LFM2, a family of Liquid Foundation Models designed for efficient on-device deployment and strong task capabilities. Using hardware-in-the-loop architecture search under…
Parallelization of Non-linear State-Space Models: Scaling Up Liquid-Resistance Liquid-Capacitance Networks for Efficient Sequence Modeling
Mónika Farsang, Ramin Hasani, Daniela Rus +1
We present LrcSSM, a recurrent model that processes long sequences as fast as today's linear state-space layers. By forcing its Jacobian matrix to be diagonal…
State-Free Inference of State-Space Models: The Transfer Function Approach
Rom N. Parnichkun, Stefano Massaroli, Alessandro Moro +10
We approach designing a state-space model for deep learning applications through its dual representation, the transfer function, and uncover a highly efficient sequence parallel in…
Leveraging Low-Rank and Sparse Recurrent Connectivity for Robust Closed-Loop Control
Neehal Tumma, Mathias Lechner, Noel Loo +2
Developing autonomous agents that can interact with changing environments is an open challenge in machine learning. Robustness is particularly important in these settings as agents…
On the Size and Approximation Error of Distilled Sets
Alaa Maalouf, Murad Tukan, Noel Loo +3
Dataset Distillation is the task of synthesizing small datasets from large ones while still retaining comparable predictive accuracy to the original uncompressed dataset. Despite s…
Dataset Distillation with Convexified Implicit Gradients
Noel Loo, Ramin Hasani, Mathias Lechner +1
We propose a new dataset distillation algorithm using reparameterization and convexification of implicit gradients (RCIG), that substantially improves the state-of-the-art. To this…
Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation
Noel Loo, Ramin Hasani, Mathias Lechner +2
Modern deep learning requires large volumes of data, which could contain sensitive or private information that cannot be leaked. Recent work has shown for homogeneous neural networ…
Efficient Dataset Distillation Using Random Feature Approximation
Noel Loo, Ramin Hasani, Alexander Amini +1
Dataset distillation compresses large datasets into smaller synthetic coresets which retain performance with the aim of reducing the storage and computational burden of processing…
Evolution of Neural Tangent Kernels under Benign and Adversarial Training
Noel Loo, Ramin Hasani, Alexander Amini +1
Two key challenges facing modern deep learning are mitigating deep networks' vulnerability to adversarial attacks and understanding deep learning's generalization capabilities. Tow…
PyHopper -- Hyperparameter optimization
Mathias Lechner, Ramin Hasani, Philipp Neubauer +2
Hyperparameter tuning is a fundamental aspect of machine learning research. Setting up the infrastructure for systematic optimization of hyperparameters can take a significant amou…
Interpreting Neural Policies with Disentangled Tree Representations
Tsun-Hsuan Wang, Wei Xiao, Tim Seyde +2
The advancement of robots, particularly those functioning in complex human-centric environments, relies on control solutions that are driven by machine learning. Understanding how…
On the Forward Invariance of Neural ODEs
Wei Xiao, Tsun-Hsuan Wang, Ramin Hasani +4
We propose a new method to ensure neural ordinary differential equations (ODEs) satisfy output specifications by using invariance set propagation. Our approach uses a class of cont…
Liquid Structural State-Space Models
Ramin Hasani, Mathias Lechner, Tsun-Hsuan Wang +3
A proper parametrization of state transition matrices of linear state-space models (SSMs) followed by standard nonlinearities enables them to efficiently learn representations from…
Entangled Residual Mappings
Mathias Lechner, Ramin Hasani, Zahra Babaiee +4
Residual mappings have been shown to perform representation learning in the first layers and iterative feature refinement in higher layers. This interplay, combined with their stab…
BarrierNet: A Safety-Guaranteed Layer for Neural Networks
Wei Xiao, Ramin Hasani, Xiao Li +1
This paper introduces differentiable higher-order control barrier functions (CBF) that are end-to-end trainable together with learning systems. CBFs are usually overly conservative…
GoTube: Scalable Stochastic Verification of Continuous-Depth Models
Sophie Gruenbacher, Mathias Lechner, Ramin Hasani +4
We introduce a new stochastic verification algorithm that formally quantifies the behavioral robustness of any time-continuous process formulated as a continuous-depth model. Our a…
Closed-form Continuous-time Neural Models
Ramin Hasani, Mathias Lechner, Alexander Amini +5
Continuous-time neural processes are performant sequential decision-makers that are built by differential equations (DE). However, their expressive power when they are deployed on…
Sparse Flows: Pruning Continuous-depth Models
Lucas Liebenwein, Ramin Hasani, Alexander Amini +1
Continuous deep learning architectures enable learning of flexible probabilistic models for predictive modeling as neural ordinary differential equations (ODEs), and for generative…
Causal Navigation by Continuous-time Neural Networks
Charles Vorbach, Ramin Hasani, Alexander Amini +2
Imitation learning enables high-fidelity, vision-based learning of policies within rich, photorealistic environments. However, such techniques often rely on traditional discrete-ti…
Adversarial Training is Not Ready for Robot Learning
Mathias Lechner, Ramin Hasani, Radu Grosu +2
Adversarial training is an effective method to train deep learning models that are resilient to norm-bounded perturbations, with the cost of nominal performance drop. While adversa…
Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing
Axel Brunnbauer, Luigi Berducci, Andreas Brandstätter +4
World models learn behaviors in a latent imagination space to enhance the sample-efficiency of deep reinforcement learning (RL) algorithms. While learning world models for high-dim…