228 citations · 1.5k across the 123 of their papers we have counts for
24 papers · 1 filter
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
Yafei Hu, Quanting Xie, Vidhi Jain +20
Building general-purpose robots that operate seamlessly in any environment, with any object, and utilizing various skills to complete diverse tasks has been a long-standing goal in…
ConstraintMatch for Semi-constrained Clustering
Jann Goschenhofer, Bernd Bischl, Zsolt Kira
Constrained clustering allows the training of classification models using pairwise constraints only, which are weak and relatively easy to mine, while still yielding full-supervisi…
DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets
Yash Jain, Harkirat Behl, Zsolt Kira +1
Construction of a universal detector poses a crucial question: How can we most effectively train a model on a large mixture of datasets? The answer lies in learning dataset-specifi…
Continual Diffusion with STAMINA: STack-And-Mask INcremental Adapters
James Seale Smith, Yen-Chang Hsu, Zsolt Kira +2
Recent work has demonstrated a remarkable ability to customize text-to-image diffusion models to multiple, fine-grained concepts in a sequential (i.e., continual) manner while only…
Fast Trainable Projection for Robust Fine-Tuning
Junjiao Tian, Yen-Cheng Liu, James Seale Smith +1
Robust fine-tuning aims to achieve competitive in-distribution (ID) performance while maintaining the out-of-distribution (OOD) robustness of a pre-trained model when transferring…
Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
Xavier Puig, Eric Undersander, Andrew Szot +20
We present Habitat 3.0: a simulation platform for studying collaborative human-robot tasks in home environments. Habitat 3.0 offers contributions across three dimensions: (1) Accur…
FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects
Mayank Lunayach, Sergey Zakharov, Dian Chen +3
In this work, we address the challenging task of 3D object recognition without the reliance on real-world 3D labeled data. Our goal is to predict the 3D shape, size, and 6D pose of…
Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models
Benjamin Hoover, Hendrik Strobelt, Dmitry Krotov +3
The generative process of Diffusion Models (DMs) has recently set state-of-the-art on many AI generation benchmarks. Though the generative process is traditionally understood as an…
LatentDR: Improving Model Generalization Through Sample-Aware Latent Degradation and Restoration
Ran Liu, Sahil Khose, Jingyun Xiao +4
Despite significant advances in deep learning, models often struggle to generalize well to new, unseen domains, especially when training data is limited. To address this challenge,…
NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes
Muhammad Zubair Irshad, Sergey Zakharov, Katherine Liu +5
Recent implicit neural representations have shown great results for novel view synthesis. However, existing methods require expensive per-scene optimization from many views hence l…
Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusion
Junjiao Tian, Lavisha Aggarwal, Andrea Colaco +2
Producing quality segmentation masks for images is a fundamental problem in computer vision. Recent research has explored large-scale supervised training to enable zero-shot segmen…
HomeRobot: Open-Vocabulary Mobile Manipulation
Sriram Yenamandra, Arun Ramachandran, Karmesh Yadav +15
HomeRobot (noun): An affordable compliant robot that navigates homes and manipulates a wide range of objects in order to complete everyday tasks. Open-Vocabulary Mobile Manipulatio…
Continual Adaptation of Vision Transformers for Federated Learning
Shaunak Halbe, James Seale Smith, Junjiao Tian +1
In this paper, we focus on the important yet understudied problem of Continual Federated Learning (CFL), where a server communicates with a set of clients to incrementally learn ne…
Adaptive Coordination in Social Embodied Rearrangement
Andrew Szot, Unnat Jain, Dhruv Batra +3
We present the task of "Social Rearrangement", consisting of cooperative everyday tasks like setting up the dinner table, tidying a house or unpacking groceries in a simulated mult…
HAAV: Hierarchical Aggregation of Augmented Views for Image Captioning
Chia-Wen Kuo, Zsolt Kira
A great deal of progress has been made in image captioning, driven by research into how to encode the image using pre-trained models. This includes visual encodings (e.g. image gri…
CLIP-GCD: Simple Language Guided Generalized Category Discovery
Rabah Ouldnoughi, Chia-Wen Kuo, Zsolt Kira
Generalized Category Discovery (GCD) requires a model to both classify known categories and cluster unknown categories in unlabeled data. Prior methods leveraged self-supervised pr…
We Need to Talk: Identifying and Overcoming Communication-Critical Scenarios for Self-Driving
Nathaniel Moore Glaser, Zsolt Kira
In this work, we consider the task of collision-free trajectory planning for connected self-driving vehicles. We specifically consider communication-critical situations--situations…
Training Energy-Based Normalizing Flow with Score-Matching Objectives
Chen-Hao Chao, Wei-Fang Sun, Yen-Chang Hsu +2
In this paper, we establish a connection between the parameterization of flow-based and energy-based generative models, and present a new flow-based modeling approach called energy…
Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentation
Harsh Maheshwari, Yen-Cheng Liu, Zsolt Kira
Using multiple spatial modalities has been proven helpful in improving semantic segmentation performance. However, there are several real-world challenges that have yet to be addre…
Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA
James Seale Smith, Yen-Chang Hsu, Lingyu Zhang +4
Recent works demonstrate a remarkable ability to customize text-to-image diffusion models while only providing a few example images. What happens if you try to customize such model…
BC-IRL: Learning Generalizable Reward Functions from Demonstrations
Andrew Szot, Amy Zhang, Dhruv Batra +2
How well do reward functions learned with inverse reinforcement learning (IRL) generalize? We illustrate that state-of-the-art IRL algorithms, which maximize a maximum-entropy obje…
Trainable Projected Gradient Method for Robust Fine-tuning
Junjiao Tian, Xiaoliang Dai, Chih-Yao Ma +3
Recent studies on transfer learning have shown that selectively fine-tuning a subset of layers or customizing different learning rates for each layer can greatly improve robustness…
OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNav
Karmesh Yadav, Arjun Majumdar, Ram Ramrakhya +5
We present a single neural network architecture composed of task-agnostic components (ViTs, convolutions, and LSTMs) that achieves state-of-art results on both the ImageNav ("go to…
Communication-Critical Planning via Multi-Agent Trajectory Exchange
Nathaniel Moore Glaser, Zsolt Kira
This paper addresses the task of joint multi-agent perception and planning, especially as it relates to the real-world challenge of collision-free navigation for connected self-dri…