Publications (8)
ORGANA: A Robotic Assistant for Automated Chemistry Experimentation and Characterization
Kourosh Darvish, Marta Skreta, Yuchi Zhao +9
Chemistry experiments can be resource- and labor-intensive, often requiring manual tasks like polishing electrodes in electrochemistry. Traditional lab automation infrastructure fa…
Predicting 3D shapes, masks, and properties of materials, liquids, and objects inside transparent containers, using the TransProteus CGI dataset
Sagi Eppel, Haoping Xu, Yi Ru Wang +1
We present TransProteus, a dataset, and methods for predicting the 3D structure, masks, and properties of materials, liquids, and objects inside transparent vessels from a single i…
Chemistry Lab Automation via Constrained Task and Motion Planning
Naruki Yoshikawa, Andrew Zou Li, Kourosh Darvish +6
Chemists need to perform many laborious and time-consuming experiments in the lab to discover and understand the properties of new materials. To support and accelerate this process…
Discovering Robotic Interaction Modes with Discrete Representation Learning
Liquan Wang, Ankit Goyal, Haoping Xu +1
Human actions manipulating articulated objects, such as opening and closing a drawer, can be categorized into multiple modalities we define as interaction modes. Traditional robot…
Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation
Lingfeng Zhang, Yuecheng Liu, Zhanguang Zhang +16
Recent advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) have made them powerful tools in embodied navigation, enabling agents to leverage commonsense…
MVTrans: Multi-View Perception of Transparent Objects
Yi Ru Wang, Yuchi Zhao, Haoping Xu +4
Transparent object perception is a crucial skill for applications such as robot manipulation in household and laboratory settings. Existing methods utilize RGB-D or stereo inputs t…
Computer vision for liquid samples in hospitals and medical labs using hierarchical image segmentation and relations prediction
Sagi Eppel, Haoping Xu, Alan Aspuru-Guzik
This work explores the use of computer vision for image segmentation and classification of medical fluid samples in transparent containers (for example, tubes, syringes, infusion b…
Seeing Glass: Joint Point Cloud and Depth Completion for Transparent Objects
Haoping Xu, Yi Ru Wang, Sagi Eppel +3
The basis of many object manipulation algorithms is RGB-D input. Yet, commodity RGB-D sensors can only provide distorted depth maps for a wide range of transparent objects due ligh…