9 papers
Audio-Visual Camera Pose Estimation with Passive Scene Sounds and In-the-Wild Video
Daniel Adebi, Sagnik Majumder, Kristen Grauman
Understanding camera motion is a fundamental problem in embodied perception and 3D scene understanding. While visual methods have advanced rapidly, they often struggle under visual…
Materialistic RIR: Material Conditioned Realistic RIR Generation
Mahnoor Fatima Saad, Sagnik Majumder, Kristen Grauman +1
Rings like gold, thuds like wood! The sound we hear in a scene is shaped not only by the spatial layout of the environment but also by the materials of the objects and surfaces wit…
Mash, Spread, Slice! Learning to Manipulate Object States via Visual Spatial Progress
Priyanka Mandikal, Jiaheng Hu, Shivin Dass +3
Most robot manipulation focuses on changing the kinematic state of objects: picking, placing, opening, or rotating them. However, a wide range of real-world manipulation tasks invo…
MistExit: Learning to Exit for Early Mistake Detection in Procedural Videos
Sagnik Majumder, Anish Nethi, Ziad Al-Halah +1
We introduce the task of early mistake detection in video, where the goal is to determine whether a keystep in a procedural activity is performed correctly while observing as littl…
Switch-a-View: View Selection Learned from Unlabeled In-the-wild Videos
Sagnik Majumder, Tushar Nagarajan, Ziad Al-Halah +1
We introduce SWITCH-A-VIEW, a model that learns to automatically select the viewpoint to display at each timepoint when creating a how-to video. The key insight of our approach is…
Which Viewpoint Shows it Best? Language for Weakly Supervising View Selection in Multi-view Instructional Videos
Sagnik Majumder, Tushar Nagarajan, Ziad Al-Halah +2
Given a multi-view video, which viewpoint is most informative for a human observer? Existing methods rely on heuristics or expensive "best-view" supervision to answer this question…