Publications (8)
RealImpact: A Dataset of Impact Sound Fields for Real Objects
Samuel Clarke, Ruohan Gao, Mason Wang +5
Objects make unique sounds under different perturbations, environment conditions, and poses relative to the listener. While prior works have modeled impact sounds and sound propaga…
X-Capture: An Open-Source Portable Device for Multi-Sensory Learning
Samuel Clarke, Suzannah Wistreich, Yanjie Ze +1
Understanding objects through multiple sensory modalities is fundamental to human perception, enabling cross-sensory integration and richer comprehension. For AI and robotic system…
Hearing Anything Anywhere
Mason Wang, Ryosuke Sawata, Samuel Clarke +3
Recent years have seen immense progress in 3D computer vision and computer graphics, with emerging tools that can virtualize real-world 3D environments for numerous Mixed Reality (…
Knowledge-based learning in Text-RAG and Image-RAG
Alexander Shim, Khalil Saieh, Samuel Clarke
This research analyzed and compared the multi-modal approach in the Vision Transformer(EVA-ViT) based image encoder with the LlaMA or ChatGPT LLM to reduce the hallucination proble…
ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer
Ruohan Gao, Zilin Si, Yen-Yu Chang +5
Objects play a crucial role in our everyday activities. Though multisensory object-centric learning has shown great potential lately, the modeling of objects in prior work is rathe…
DexSkin: High-Coverage Conformable Robotic Skin for Learning Contact-Rich Manipulation
Suzannah Wistreich, Baiyu Shi, Stephen Tian +5
Human skin provides a rich tactile sensing stream, localizing intentional and unintentional contact events over a large and contoured region. Replicating these tactile sensing capa…
SoundCam: A Dataset for Finding Humans Using Room Acoustics
Mason Wang, Samuel Clarke, Jui-Hsien Wang +2
A room's acoustic properties are a product of the room's geometry, the objects within the room, and their specific positions. A room's acoustic properties can be characterized by i…
RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools
Haochen Shi, Huazhe Xu, Samuel Clarke +2
Humans excel in complex long-horizon soft body manipulation tasks via flexible tool use: bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often reg…