14 papers
From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation
Rit Gangopadhyay, Alex Wong
Vision foundation models are capable of generalizing across 3-dimensional (3D) scenes with high-fidelity estimates; their empirical success can be attributed to training on large-s…
VisTa3D: A Dataset and Benchmark for Thin Object Reconstruction from Vision, Tactile, and 3D Point Clouds
Shania Guo, Yeongsik Seo, Andrew Fu +7
State-of-the-art 3D reconstruction models, whether from visual, range, or both, tend to underperform on thin objects. This is partially due to the small amount of space such object…
UniTac: A Unified Multimodal Model for Cross-Sensor Tactile Understanding and Generation
Jiahang Tu, Fengyu Yang, Chenyang Ma +8
Unified multimodal models (UMMs) have shown great promise in integrating understanding and generation across diverse modalities. However, existing research rarely extends this para…
ETA: Energy-based Test-time Adaptation for Depth Completion
Younjoon Chung, Hyoungseob Park, Patrick Rim +7
We propose a method for test-time adaptation of pretrained depth completion models. Depth completion models, trained on some ``source'' data, often predict erroneous outputs when t…
Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens
Rit Gangopadhyay, Jung-Hee Kim, Xien Chen +3
We propose a method to extend foundational monocular depth estimators (FMDEs), trained on perspective images, to fisheye images. Despite being trained on tens of millions of images…
An Information-Theoretic Framework for Feature Construction in Out-of-Distribution Detection
Sudeepta Mondal, Xinyi, Xie +2
We present a theory for the construction of out-of-distribution (OOD) detection features for neural networks. We introduce random features for OOD through a novel information-theor…
ODE-GS: Latent ODEs for Dynamic Scene Extrapolation with 3D Gaussian Splatting
Daniel Wang, Patrick Rim, Tian Tian +3
We introduce ODE-GS, a novel approach that integrates 3D Gaussian Splatting with latent neural ordinary differential equations (ODEs) to enable future extrapolation of dynamic 3D s…
Progressive Test Time Energy Adaptation for Medical Image Segmentation
Xiaoran Zhang, Byung-Woo Hong, Hyoungseob Park +5
We propose a model-agnostic, progressive test-time energy adaptation approach for medical image segmentation. Maintaining model performance across diverse medical datasets is chall…
ProtoDepth: Unsupervised Continual Depth Completion with Prototypes
Patrick Rim, Hyoungseob Park, S. Gangopadhyay +3
We present ProtoDepth, a novel prototype-based approach for continual learning of unsupervised depth completion, the multimodal 3D reconstruction task of predicting dense depth map…
Radar-Guided Polynomial Fitting for Metric Depth Estimation
Patrick Rim, Hyoungseob Park, Vadim Ezhov +2
We propose POLAR, a novel radar-guided depth estimation method that introduces polynomial fitting to efficiently transform scaleless depth predictions from pretrained monocular dep…
UnCLe: Benchmarking Unsupervised Continual Learning for Depth Completion
Xien Chen, Rit Gangopadhyay, Michael Chu +3
We propose UnCLe, the first standardized benchmark for Unsupervised Continual Learning of a multimodal 3D reconstruction task: Depth completion aims to infer a dense depth map from…
NeuroBind: Towards Unified Multimodal Representations for Neural Signals
Fengyu Yang, Chao Feng, Daniel Wang +8
Understanding neural activity and information representation is crucial for advancing knowledge of brain function and cognition. Neural activity, measured through techniques like e…
Test-Time Adaptation for Depth Completion
Hyoungseob Park, Anjali Gupta, Alex Wong
It is common to observe performance degradation when transferring models trained on some (source) datasets to target testing data due to a domain gap between them. Existing methods…
Binding Touch to Everything: Learning Unified Multimodal Tactile Representations
Fengyu Yang, Chao Feng, Ziyang Chen +8
The ability to associate touch with other modalities has huge implications for humans and computational systems. However, multimodal learning with touch remains challenging due to…