Publications (36)
LFM2 Technical Report
Alexander Amini, Anna Banaszak, Harold Benoit +30
We present LFM2, a family of Liquid Foundation Models designed for efficient on-device deployment and strong task capabilities. Using hardware-in-the-loop architecture search under…
Toward Efficient Visual Gyroscopes: Spherical Moments, Harmonics Filtering, and Masking Techniques for Spherical Camera Applications
Yao Du, Carlos M. Mateo, Mirjana Maras +5
Unlike a traditional gyroscope, a visual gyroscope estimates camera rotation through images. The integration of omnidirectional cameras, offering a larger field of view compared to…
Liquid Time-constant Networks
Ramin Hasani, Mathias Lechner, Alexander Amini +2
We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of…
The Impact of Social Segregation on Human Mobility in Developing and Urbanized Regions
Alexander Amini, Kevin Kung, Chaogui Kang +2
This study leverages mobile phone data to analyze human mobility patterns in developing countries, especially in comparison to more industrialized countries. Developing regions, su…
Uncertainty-aware Language Modeling for Selective Question Answering
Qi Yang, Shreya Ravikumar, Fynn Schmitt-Ulms +9
We present an automatic large language model (LLM) conversion approach that produces uncertainty-aware LLMs capable of estimating uncertainty with every prediction. Our approach is…
Variational End-to-End Navigation and Localization
Alexander Amini, Guy Rosman, Sertac Karaman +1
Deep learning has revolutionized the ability to learn "end-to-end" autonomous vehicle control directly from raw sensory data. While there have been recent extensions to handle form…
BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation
Zhijian Liu, Haotian Tang, Alexander Amini +4
Multi-sensor fusion is essential for an accurate and reliable autonomous driving system. Recent approaches are based on point-level fusion: augmenting the LiDAR point cloud with ca…
Differentiable Control Barrier Functions for Vision-based End-to-End Autonomous Driving
Wei Xiao, Tsun-Hsuan Wang, Makram Chahine +3
Guaranteeing safety of perception-based learning systems is challenging due to the absence of ground-truth state information unlike in state-aware control scenarios. In this paper,…
Liquid Time-constant Recurrent Neural Networks as Universal Approximators
Ramin M. Hasani, Mathias Lechner, Alexander Amini +2
In this paper, we introduce the notion of liquid time-constant (LTC) recurrent neural networks (RNN)s, a subclass of continuous-time RNNs, with varying neuronal time-constant reali…
Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models
Tsun-Hsuan Wang, Alaa Maalouf, Wei Xiao +5
As autonomous driving technology matures, end-to-end methodologies have emerged as a leading strategy, promising seamless integration from perception to control via deep learning.…
VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles
Alexander Amini, Tsun-Hsuan Wang, Igor Gilitschenski +5
Simulation has the potential to transform the development of robust algorithms for mobile agents deployed in safety-critical scenarios. However, the poor photorealism and lack of d…
Sparse Flows: Pruning Continuous-depth Models
Lucas Liebenwein, Ramin Hasani, Alexander Amini +1
Continuous deep learning architectures enable learning of flexible probabilistic models for predictive modeling as neural ordinary differential equations (ODEs), and for generative…
Gaussian Splatting to Real World Flight Navigation Transfer with Liquid Networks
Alex Quach, Makram Chahine, Alexander Amini +2
Simulators are powerful tools for autonomous robot learning as they offer scalable data generation, flexible design, and optimization of trajectories. However, transferring behavio…
Spatial Uncertainty Sampling for End-to-End Control
Alexander Amini, Ava Soleimany, Sertac Karaman +1
End-to-end trained neural networks (NNs) are a compelling approach to autonomous vehicle control because of their ability to learn complex tasks without manual engineering of rule-…
Accelerated Convolutions for Efficient Multi-Scale Time to Contact Computation in Julia
Alexander Amini, Berthold Horn, Alan Edelman
Convolutions have long been regarded as fundamental to applied mathematics, physics and engineering. Their mathematical elegance allows for common tasks such as numerical different…
Exploring Latent Pathways: Enhancing the Interpretability of Autonomous Driving with a Variational Autoencoder
Anass Bairouk, Mirjana Maras, Simon Herlin +5
Autonomous driving presents a complex challenge, which is usually addressed with artificial intelligence models that are end-to-end or modular in nature. Within the landscape of mo…
Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learning
Mathias Lechner, Alexander Amini, Daniela Rus +1
Adversarial training (i.e., training on adversarially perturbed input data) is a well-studied method for making neural networks robust to potential adversarial attacks during infer…
Efficient Dataset Distillation Using Random Feature Approximation
Noel Loo, Ramin Hasani, Alexander Amini +1
Dataset distillation compresses large datasets into smaller synthetic coresets which retain performance with the aim of reducing the storage and computational burden of processing…
Evolution of Neural Tangent Kernels under Benign and Adversarial Training
Noel Loo, Ramin Hasani, Alexander Amini +1
Two key challenges facing modern deep learning are mitigating deep networks' vulnerability to adversarial attacks and understanding deep learning's generalization capabilities. Tow…
Learning Interactive Driving Policies via Data-driven Simulation
Tsun-Hsuan Wang, Alexander Amini, Wilko Schwarting +3
Data-driven simulators promise high data-efficiency for driving policy learning. When used for modelling interactions, this data-efficiency becomes a bottleneck: Small underlying d…
Learning Stability Attention in Vision-based End-to-end Driving Policies
Tsun-Hsuan Wang, Wei Xiao, Makram Chahine +3
Modern end-to-end learning systems can learn to explicitly infer control from perception. However, it is difficult to guarantee stability and robustness for these systems since the…
Efficient and Robust LiDAR-Based End-to-End Navigation
Zhijian Liu, Alexander Amini, Sibo Zhu +3
Deep learning has been used to demonstrate end-to-end neural network learning for autonomous vehicle control from raw sensory input. While LiDAR sensors provide reliably accurate i…
Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks
Ramin M. Hasani, Alexander Amini, Mathias Lechner +3
In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a s…
In-Place Tokenizer Expansion for Pre-trained LLMs
Jimmy T. H. Smith, Tarek Dakhran, Alberto Cabrera +7
The paper proposes an in‑place tokenizer expansion method that continues a pre‑trained model’s BPE merges on multilingual data, reuses existing token embeddings, and initializes ne…
MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery
Maksim Kuznetsov, Zulfat Miftahutdinov, Rim Shayakhmetov +17
General-purpose large language models (LLMs) that rely on in-context learning do not reliably deliver the scientific understanding and performance required for drug discovery tasks…
Causal Navigation by Continuous-time Neural Networks
Charles Vorbach, Ramin Hasani, Alexander Amini +2
Imitation learning enables high-fidelity, vision-based learning of policies within rich, photorealistic environments. However, such techniques often rely on traditional discrete-ti…
Deep Evidential Regression
Alexander Amini, Wilko Schwarting, Ava Soleimany +1
Deterministic neural networks (NNs) are increasingly being deployed in safety critical domains, where calibrated, robust, and efficient measures of uncertainty are crucial. In this…
Are All Vision Models Created Equal? A Study of the Open-Loop to Closed-Loop Causality Gap
Mathias Lechner, Ramin Hasani, Alexander Amini +3
There is an ever-growing zoo of modern neural network models that can efficiently learn end-to-end control from visual observations. These advanced deep models, ranging from convol…
Liquid Structural State-Space Models
Ramin Hasani, Mathias Lechner, Tsun-Hsuan Wang +3
A proper parametrization of state transition matrices of linear state-space models (SSMs) followed by standard nonlinearities enables them to efficiently learn representations from…
Multi-Agent Robotic Control with Onboard Vision-Language Models
Kajetan RachwaÅ, Maciej Majek, BartÅomiej Boczek +6
Vision Language Models (VLMs) and Vision Language Action (VLA) models have shown promise in robotic control. Yet, they face significant challenges regarding explainability, general…
STAR: Synthesis of Tailored Architectures
Armin W. Thomas, Rom Parnichkun, Alexander Amini +2
Iterative improvement of model architectures is fundamental to deep learning: Transformers first enabled scaling, and recent advances in model hybridization have pushed the quality…
Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation
Noel Loo, Ramin Hasani, Mathias Lechner +2
Modern deep learning requires large volumes of data, which could contain sensitive or private information that cannot be leaked. Recent work has shown for homogeneous neural networ…
Analysis of pedestrian behaviors through non-invasive Bluetooth monitoring
Yuji Yoshimura, Alexander Amini, Stanislav Sobolevsky +2
This paper analyzes pedestrians' behavioral patterns in the pedestrianized shopping environment in the historical center of Barcelona, Spain. We employ a Bluetooth detection techni…
Closed-form Continuous-time Neural Models
Ramin Hasani, Mathias Lechner, Alexander Amini +5
Continuous-time neural processes are performant sequential decision-makers that are built by differential equations (DE). However, their expressive power when they are deployed on…
Capsa: A Unified Framework for Quantifying Risk in Deep Neural Networks
Sadhana Lolla, Iaroslav Elistratov, Alejandro Perez +3
The modern pervasiveness of large-scale deep neural networks (NNs) is driven by their extraordinary performance on complex problems but is also plagued by their sudden, unexpected,…
Can a Compact Neuronal Circuit Policy be Re-purposed to Learn Simple Robotic Control?
Ramin Hasani, Mathias Lechner, Alexander Amini +2
We propose a neural information processing system which is obtained by re-purposing the function of a biological neural circuit model, to govern simulated and real-world control ta…