47 citations · 83 across the 13 of their papers we have counts for
14 papers
Just Respect the Semantics: Image-Token Compression via Pre-LLM Semantic Variable-Resolution Tokenization and In-LLM Relevance Pruning
Tianhao Fu, Austin Wang, Charles Chen +5
Image-token compression must reduce computation without hurting the semantics. We introduce Varest, guided by semantic fungibility: tokens within a coherent semantic region can be…
Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models
Austin Wang, Jiaqi Han, Stefano Ermon +1
Preference optimization has emerged as an efficient alternative to online reinforcement learning from human feedback (RLHF) for aligning text-to-image diffusion models. However, ex…
SegWithU: Uncertainty as Perturbation Energy for Single-Forward-Pass Risk-Aware Medical Image Segmentation
Tianhao Fu, Austin Wang, Charles Chen +2
Reliable uncertainty estimation is critical for medical image segmentation, where automated contours feed downstream quantification and clinical decision support. Many strong uncer…
Blade: A Derivative-free Bayesian Inversion Method using Diffusion Priors
Hongkai Zheng, Austin Wang, Zihui Wu +3
Derivative-free Bayesian inversion arises in science and engineering applications, particularly when forward model is costly or infeasible to differentiate through. Existing deriva…
Towards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challenge
Sriram Yenamandra, Arun Ramachandran, Mukul Khanna +42
In order to develop robots that can effectively serve as versatile and capable home assistants, it is crucial for them to reliably perceive and interact with a wide variety of obje…
HomeRobot: Open-Vocabulary Mobile Manipulation
Sriram Yenamandra, Arun Ramachandran, Karmesh Yadav +15
HomeRobot (noun): An affordable compliant robot that navigates homes and manipulates a wide range of objects in order to complete everyday tasks. Open-Vocabulary Mobile Manipulatio…
USA-Net: Unified Semantic and Affordance Representations for Robot Memory
Benjamin Bolte, Austin Wang, Jimmy Yang +3
In order for robots to follow open-ended instructions like "go open the brown cabinet over the sink", they require an understanding of both the scene geometry and the semantics of…
Navigating to Objects Specified by Images
Jacob Krantz, Theophile Gervet, Karmesh Yadav +7
Images are a convenient way to specify which particular object instance an embodied agent should navigate to. Solving this task requires semantic visual reasoning and exploration o…
Theseus: A Library for Differentiable Nonlinear Optimization
Luis Pineda, Taosha Fan, Maurizio Monge +10
We present Theseus, an efficient application-agnostic open source library for differentiable nonlinear least squares (DNLS) optimization built on PyTorch, providing a common framew…
RB2: Robotic Manipulation Benchmarking with a Twist
Sudeep Dasari, Jianren Wang, Joyce Hong +12
Benchmarks offer a scientific way to compare algorithms using objective performance metrics. Good benchmarks have two features: (a) they should be widely useful for many research g…
Differentiable and Learnable Robot Models
Franziska Meier, Austin Wang, Giovanni Sutanto +2
Building differentiable simulations of physical processes has recently received an increasing amount of attention. Specifically, some efforts develop differentiable robotic physics…
Efficient and Interpretable Robot Manipulation with Graph Neural Networks
Yixin Lin, Austin S. Wang, Eric Undersander +1
Manipulation tasks, like loading a dishwasher, can be seen as a sequence of spatial constraints and relationships between different objects. We aim to discover these rules from dem…
Learning State-Dependent Losses for Inverse Dynamics Learning
Kristen Morse, Neha Das, Yixin Lin +3
Being able to quickly adapt to changes in dynamics is paramount in model-based control for object manipulation tasks. In order to influence fast adaptation of the inverse dynamics…
Encoding Physical Constraints in Differentiable Newton-Euler Algorithm
Giovanni Sutanto, Austin S. Wang, Yixin Lin +4
The recursive Newton-Euler Algorithm (RNEA) is a popular technique for computing the dynamics of robots. RNEA can be framed as a differentiable computational graph, enabling the dy…