4 papers
Source-Learned Reliance for Selective Test-Time Adaptation of Multimodal Time Series
Payal Mohapatra, Yueyuan Sui, Haodong Yang +3
Multimodal wearable systems must remain reliable when sensor streams become noisy or unavailable. Existing multimodal test-time adaptation (TTA) methods often assess reliability on…
Efficient Multimodal Inference through Adaptive Acquisition and Sequential Fusion
Payal Mohapatra, Haodong Yang, Yueyuan Sui +3
Multimodal systems often encode every available input, even when a subset suffices for prediction. Adaptive acquisition can reduce this cost by using predictions from incrementally…
How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions
Aditya Prakash, Benjamin Lundell, Dmitry Andreychuk +3
We tackle the novel problem of predicting 3D hand motion and contact maps (or Interaction Trajectories) given a single RGB view, action text, and a 3D contact point on the object a…
STEPs: Self-Supervised Key Step Extraction and Localization from Unlabeled Procedural Videos
Anshul Shah, Benjamin Lundell, Harpreet Sawhney +1
We address the problem of extracting key steps from unlabeled procedural videos, motivated by the potential of Augmented Reality (AR) headsets to revolutionize job training and per…