223 citations · 522 across the 34 of their papers we have counts for
34 papers
In-Place Tokenizer Expansion for Pre-trained LLMs
Jimmy T. H. Smith, Tarek Dakhran, Alberto Cabrera +7
A tokenizer fixed at the start of pre-training allocates vocabulary in proportion to the pre-training corpus, reflecting the deployment priorities at that time. When those prioriti…
Multi-Agent Robotic Control with Onboard Vision-Language Models
Kajetan Rachwał, Maciej Majek, Bartłomiej Boczek +6
Vision Language Models (VLMs) and Vision Language Action (VLA) models have shown promise in robotic control. Yet, they face significant challenges regarding explainability, general…
LFM2 Technical Report
Alexander Amini, Anna Banaszak, Harold Benoit +30
We present LFM2, a family of Liquid Foundation Models designed for efficient on-device deployment and strong task capabilities. Using hardware-in-the-loop architecture search under…
Gaussian Splatting to Real World Flight Navigation Transfer with Liquid Networks
Alex Quach, Makram Chahine, Alexander Amini +2
Simulators are powerful tools for autonomous robot learning as they offer scalable data generation, flexible design, and optimization of trajectories. However, transferring behavio…
Exploring Latent Pathways: Enhancing the Interpretability of Autonomous Driving with a Variational Autoencoder
Anass Bairouk, Mirjana Maras, Simon Herlin +5
Autonomous driving presents a complex challenge, which is usually addressed with artificial intelligence models that are end-to-end or modular in nature. Within the landscape of mo…
Toward Efficient Visual Gyroscopes: Spherical Moments, Harmonics Filtering, and Masking Techniques for Spherical Camera Applications
Yao Du, Carlos M. Mateo, Mirjana Maras +5
Unlike a traditional gyroscope, a visual gyroscope estimates camera rotation through images. The integration of omnidirectional cameras, offering a larger field of view compared to…
Uncertainty-aware Language Modeling for Selective Question Answering
Qi Yang, Shreya Ravikumar, Fynn Schmitt-Ulms +9
We present an automatic large language model (LLM) conversion approach that produces uncertainty-aware LLMs capable of estimating uncertainty with every prediction. Our approach is…
Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models
Tsun-Hsuan Wang, Alaa Maalouf, Wei Xiao +5
As autonomous driving technology matures, end-to-end methodologies have emerged as a leading strategy, promising seamless integration from perception to control via deep learning.…
Capsa: A Unified Framework for Quantifying Risk in Deep Neural Networks
Sadhana Lolla, Iaroslav Elistratov, Alejandro Perez +3
The modern pervasiveness of large-scale deep neural networks (NNs) is driven by their extraordinary performance on complex problems but is also plagued by their sudden, unexpected,…
Learning Stability Attention in Vision-based End-to-end Driving Policies
Tsun-Hsuan Wang, Wei Xiao, Makram Chahine +3
Modern end-to-end learning systems can learn to explicitly infer control from perception. However, it is difficult to guarantee stability and robustness for these systems since the…
Efficient Dataset Distillation Using Random Feature Approximation
Noel Loo, Ramin Hasani, Alexander Amini +1
Dataset distillation compresses large datasets into smaller synthetic coresets which retain performance with the aim of reducing the storage and computational burden of processing…
Evolution of Neural Tangent Kernels under Benign and Adversarial Training
Noel Loo, Ramin Hasani, Alexander Amini +1
Two key challenges facing modern deep learning are mitigating deep networks' vulnerability to adversarial attacks and understanding deep learning's generalization capabilities. Tow…
Are All Vision Models Created Equal? A Study of the Open-Loop to Closed-Loop Causality Gap
Mathias Lechner, Ramin Hasani, Alexander Amini +3
There is an ever-growing zoo of modern neural network models that can efficiently learn end-to-end control from visual observations. These advanced deep models, ranging from convol…
Liquid Structural State-Space Models
Ramin Hasani, Mathias Lechner, Tsun-Hsuan Wang +3
A proper parametrization of state transition matrices of linear state-space models (SSMs) followed by standard nonlinearities enables them to efficiently learn representations from…
BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation
Zhijian Liu, Haotian Tang, Alexander Amini +4
Multi-sensor fusion is essential for an accurate and reliable autonomous driving system. Recent approaches are based on point-level fusion: augmenting the LiDAR point cloud with ca…
Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learning
Mathias Lechner, Alexander Amini, Daniela Rus +1
Adversarial training (i.e., training on adversarially perturbed input data) is a well-studied method for making neural networks robust to potential adversarial attacks during infer…
Differentiable Control Barrier Functions for Vision-based End-to-End Autonomous Driving
Wei Xiao, Tsun-Hsuan Wang, Makram Chahine +3
Guaranteeing safety of perception-based learning systems is challenging due to the absence of ground-truth state information unlike in state-aware control scenarios. In this paper,…
Learning Interactive Driving Policies via Data-driven Simulation
Tsun-Hsuan Wang, Alexander Amini, Wilko Schwarting +3
Data-driven simulators promise high data-efficiency for driving policy learning. When used for modelling interactions, this data-efficiency becomes a bottleneck: Small underlying d…
VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles
Alexander Amini, Tsun-Hsuan Wang, Igor Gilitschenski +5
Simulation has the potential to transform the development of robust algorithms for mobile agents deployed in safety-critical scenarios. However, the poor photorealism and lack of d…
Finding quadruply imaged quasars with machine learning. I. Methods
A. Akhazhanov, A. More, A. Amini +68
Strongly lensed quadruply imaged quasars (quads) are extraordinary objects. They are very rare in the sky -- only a few tens are known to date -- and yet they provide unique inform…
Closed-form Continuous-time Neural Models
Ramin Hasani, Mathias Lechner, Alexander Amini +5
Continuous-time neural processes are performant sequential decision-makers that are built by differential equations (DE). However, their expressive power when they are deployed on…
Sparse Flows: Pruning Continuous-depth Models
Lucas Liebenwein, Ramin Hasani, Alexander Amini +1
Continuous deep learning architectures enable learning of flexible probabilistic models for predictive modeling as neural ordinary differential equations (ODEs), and for generative…
Causal Navigation by Continuous-time Neural Networks
Charles Vorbach, Ramin Hasani, Alexander Amini +2
Imitation learning enables high-fidelity, vision-based learning of policies within rich, photorealistic environments. However, such techniques often rely on traditional discrete-ti…
Efficient and Robust LiDAR-Based End-to-End Navigation
Zhijian Liu, Alexander Amini, Sibo Zhu +3
Deep learning has been used to demonstrate end-to-end neural network learning for autonomous vehicle control from raw sensory input. While LiDAR sensors provide reliably accurate i…