10 papers
Repurposing Pre-trained LLMs as High Fidelity Continuous Text Autoencoders
Arkanath Pathak, Unnat Jain, Alexander C. Berg
Next-token prediction has enabled highly fluent autoregressive language models, but it represents global structure only indirectly through sequential factorization. In contrast, hi…
Learning Beyond What Humans Can Demonstrate
Yuchen Song, Aditya Mittal, Unnat Jain
Behavior cloning for robot manipulation relies on expert demonstrations. However, for tasks that require dynamic stability, precise contact timing, or dexterous coordination, human…
TEMPO: Learning Temporal Context for Dynamic Robot Manipulation
Zhenyang Feng, Jimin Heo, Erik B. Sudderth +1
Vision-language-action (VLA) models have achieved impressive performance in quasi-static manipulation, but struggle in dynamic manipulation tasks because they operate on a single o…
Unified Visuomotor Targets: Supervising VLAs Beyond Physical Actions
Zhenyang Feng, Unnat Jain
VLA models are trained to predict robot actions from visual and language observations. This is a natural choice, but it creates a mismatch: VLMs encode rich, high-level representat…
DeVA: Decoupled Video-Action Model with physical guidance for robot policy learning
Mengqi Zhang, Sahil Khose, Simar Kareer +3
Generalizable robot manipulation requires policies that can anticipate how visual scenes evolve while executing language instructions. While recent Vision-Language-Action models be…
Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment
Dwip Dalal, Shivansh Patel, Chahit Jain +8
Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision-language-action (VLA) policies. Howe…
CRAFT: A Tendon-Driven Hand with Hybrid Hard-Soft Compliance
Leo Lin, Shivansh Patel, Jay Moon +2
We introduce CRAFT hand, a tendon-driven anthropomorphic hand with hybrid hard-soft compliance for contact-rich manipulation. The design is based on a simple idea: contact is not u…
VISOR: VIsual Spatial Object Reasoning for Language-driven Object Navigation
Francesco Taioli, Shiping Yang, Sonia Raychaudhuri +3
Language-driven object navigation requires agents to interpret natural language descriptions of target objects, which combine intrinsic and extrinsic attributes for instance recogn…
ViPRA: Video Prediction for Robot Actions
Sandeep Routray, Hengkai Pan, Unnat Jain +2
Can we turn a video prediction model into a robot policy? Videos, including those of humans or teleoperated robots, capture rich physical interactions. However, most of them lack l…
Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping
Dwip Dalal, Gautam Vashishtha, Utkarsh Mishra +6
Multimodal large language models (MLLMs) often miss small details and spatial relations in cluttered scenes, leading to errors in fine-grained perceptual grounding. We introduce At…