43 citations · 150 across the 24 of their papers we have counts for
11 papers · 1 filter
On Uni-Modal Feature Learning in Supervised Multi-Modal Learning
Chenzhuang Du, Jiaye Teng, Tingle Li +5
We abstract the features (i.e. learned representations) of multi-modal data into 1) uni-modal features, which can be learned from uni-modal training, and 2) paired features, which…
The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation
Zihui Xue, Zhengqi Gao, Sucheng Ren +1
Crossmodal knowledge distillation (KD) extends traditional knowledge distillation to the area of multimodal learning and demonstrates great success in various applications. To achi…
Learning Visual Styles from Audio-Visual Associations
Tingle Li, Yichen Liu, Andrew Owens +1
From the patter of rain to the crunch of snow, the sounds we hear often convey the visual textures that appear within a scene. In this paper, we present a method for learning visua…
MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries
Tianyuan Zhang, Xuanyao Chen, Yue Wang +2
Accurate and consistent 3D tracking from multiple cameras is a key component in a vision-based autonomous driving system. It involves modeling 3D dynamic objects in complex scenes…
Training-Free Robust Multimodal Learning via Sample-Wise Jacobian Regularization
Zhengqi Gao, Sucheng Ren, Zihui Xue +2
Multimodal fusion emerges as an appealing technique to improve model performances on many tasks. Nevertheless, the robustness of such fusion methods is rarely involved in the prese…
Self-supervision through Random Segments with Autoregressive Coding (RandSAC)
Tianyu Hua, Yonglong Tian, Sucheng Ren +3
Inspired by the success of self-supervised autoregressive representation learning in natural language (GPT and its variants), and advances in recent visual architecture design with…
FUTR3D: A Unified Sensor Fusion Framework for 3D Detection
Xuanyao Chen, Tianyuan Zhang, Yue Wang +2
Sensor fusion is an essential topic in many perception systems, such as autonomous driving and robotics. Existing multi-modal 3D detection models usually involve customized designs…
Co-advise: Cross Inductive Bias Distillation
Sucheng Ren, Zhengqi Gao, Tianyu Hua +4
Transformers recently are adapted from the community of natural language processing as a promising substitute of convolution-based neural networks for visual learning tasks. Howeve…
Multimodal Knowledge Expansion
Zihui Xue, Sucheng Ren, Zhengqi Gao +1
The popularity of multimodal sensors and the accessibility of the Internet have brought us a massive amount of unlabeled multimodal data. Since existing datasets and well-trained m…
Predictive Visual Tracking: A New Benchmark and Baseline Approach
Bowen Li, Yiming Li, Junjie Ye +2
As a crucial robotic perception capability, visual tracking has been intensively studied recently. In the real-world scenarios, the onboard processing time of the image streams ine…
LID 2020: The Learning from Imperfect Data Challenge Results
Yunchao Wei, Shuai Zheng, Ming-Ming Cheng +32
Learning from imperfect data becomes an issue in many industrial applications after the research community has made profound progress in supervised learning from perfectly annotate…