6 papers
Revisiting KRISP: A Lightweight Reproduction and Analysis of Knowledge-Enhanced Vision-Language Models
Souradeep Dutta, Keshav Bulia, Neena S Nair
Facebook AI Research introduced KRISP [4], which integrates structured external knowledge into pipelines for vision-language reasoning. Despite its effectiveness, the original mode…
RICL: Adding In-Context Adaptability to Pre-Trained Vision-Language-Action Models
Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman +1
Multi-task ``vision-language-action'' (VLA) models have recently demonstrated increasing promise as generalist foundation models for robotics, achieving non-trivial performance out…
Safety Monitoring for Learning-Enabled Cyber-Physical Systems in Out-of-Distribution Scenarios
Vivian Lin, Ramneet Kaur, Yahan Yang +6
The safety of learning-enabled cyber-physical systems is compromised by the well-known vulnerabilities of deep neural networks to out-of-distribution (OOD) inputs. Existing literat…
Distributionally Robust Statistical Verification with Imprecise Neural Networks
Souradeep Dutta, Michele Caprio, Vivian Lin +5
A particularly challenging problem in AI safety is providing guarantees on the behavior of high-dimensional autonomous systems. Verification approaches centered around reachability…
REGENT: A Retrieval-Augmented Generalist Agent That Can Act In-Context in New Environments
Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman +1
Building generalist agents that can rapidly adapt to new environments is a key challenge for deploying AI in the digital and real worlds. Is scaling current agent architectures the…
Credal Bayesian Deep Learning
Michele Caprio, Souradeep Dutta, Kuk Jin Jang +4
Uncertainty quantification and robustness to distribution shifts are important goals in machine learning and artificial intelligence. Although Bayesian Neural Networks (BNNs) allow…