349 citations · 2.5k across the 74 of their papers we have counts for
75 papers
ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
Qiao Gu, Alihusein Kuwajerwala, Sacha Morin +13
For robots to perform a wide variety of tasks, they require a 3D representation of the world that is semantically rich, yet compact and efficient for task-driven perception and pla…
Aligning Large Multimodal Models with Factually Augmented RLHF
Zhiqing Sun, Sheng Shen, Shengcao Cao +9
Large Multimodal Models (LMM) are built across modalities and the misalignment between two modalities can result in "hallucination", generating textual outputs that are not grounde…
ModuleFormer: Modularity Emerges from Mixture-of-Experts
Yikang Shen, Zheyu Zhang, Tianyou Cao +3
Large Language Models (LLMs) have achieved remarkable results. However, existing models are expensive to train and deploy, and it is also difficult to expand their knowledge beyond…
Learning Situation Hyper-Graphs for Video Question Answering
Aisha Urooj Khan, Hilde Kuehne, Bo Wu +5
Answering questions about complex situations in videos requires not only capturing the presence of actors, objects, and their relations but also the evolution of these relationship…
CLAWSAT: Towards Both Robust and Accurate Code Models
Jinghan Jia, Shashank Srikant, Tamara Mitrovska +4
We integrate contrastive learning (CL) with adversarial learning to co-optimize the robustness and accuracy of code models. Different from existing works, we show that code obfusca…
Masked Motion Encoding for Self-Supervised Video Representation Learning
Xinyu Sun, Peihao Chen, Liangwei Chen +4
How to learn discriminative video representation from unlabeled videos is challenging but crucial for video analysis. The latest attempts seek to learn a representation model by pr…
Gait Recognition in the Wild with Multi-hop Temporal Switch
Jinkai Zheng, Xinchen Liu, Xiaoyan Gu +5
Existing studies for gait recognition are dominated by in-the-lab scenarios. Since people live in real-world senses, gait recognition in the wild is a more practical problem that h…
Prototype-Guided Continual Adaptation for Class-Incremental Unsupervised Domain Adaptation
Hongbin Lin, Yifan Zhang, Zhen Qiu +4
This paper studies a new, practical but challenging problem, called Class-Incremental Unsupervised Domain Adaptation (CI-UDA), where the labeled source domain contains all classes,…
On-Device Training Under 256KB Memory
Ji Lin, Ligeng Zhu, Wei-Ming Chen +3
On-device training enables the model to adapt to new data collected from the sensors by fine-tuning a pre-trained model. Users can benefit from customized AI models without having…
SNAKE: Shape-aware Neural 3D Keypoint Field
Chengliang Zhong, Peixing You, Xiaoxue Chen +6
Detecting 3D keypoints from point clouds is important for shape reconstruction, while this work investigates the dual question: can shape reconstruction benefit 3D keypoint detecti…
EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction
Han Cai, Junyan Li, Muyan Hu +2
High-resolution dense prediction enables many appealing real-world applications, such as computational photography, autonomous driving, etc. However, the vast computational cost ma…
AutoGPart: Intermediate Supervision Search for Generalizable 3D Part Segmentation
Xueyi Liu, Xiaomeng Xu, Anyi Rao +2
Training a generalizable 3D part segmentation network is quite challenging but of great importance in real-world applications. To tackle this problem, some works design task-specif…
Graph Convolutional Module for Temporal Action Localization in Videos
Runhao Zeng, Wenbing Huang, Mingkui Tan +4
Temporal action localization has long been researched in computer vision. Existing state-of-the-art action localization methods divide each video into multiple action units (i.e.,…
When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?
Lijie Fan, Sijia Liu, Pin-Yu Chen +2
Contrastive learning (CL) can learn generalizable feature representations and achieve the state-of-the-art performance of downstream tasks by finetuning a linear classifier on top…
MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
Ji Lin, Wei-Ming Chen, Han Cai +2
Tiny deep learning on microcontroller units (MCUs) is challenging due to the limited memory size. We find that the memory bottleneck is due to the imbalanced memory distribution in…
Network Augmentation for Tiny Deep Learning
Han Cai, Chuang Gan, Ji Lin +1
We introduce Network Augmentation (NetAug), a new training method for improving the performance of tiny neural networks. Existing regularization techniques (e.g., data augmentation…
TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Device
Ji Lin, Chuang Gan, Kuan Wang +1
The explosive growth in video streaming requires video understanding at high accuracy and low computation cost. Conventional 2D CNNs are computationally cheap but cannot capture te…
Self-supervised Audiovisual Representation Learning for Remote Sensing Data
Konrad Heidler, Lichao Mou, Di Hu +5
Many current deep learning approaches make extensive use of backbone networks pre-trained on large datasets like ImageNet, which are then fine-tuned to perform a certain task. In r…
Certifiably Robust Interpretation via Renyi Differential Privacy
Ao Liu, Xiaoyu Chen, Sijia Liu +2
Motivated by the recent discovery that the interpretation maps of CNNs could easily be manipulated by adversarial attacks against network interpretability, we study the problem of…
Global Rhythm Style Transfer Without Text Transcriptions
Kaizhi Qian, Yang Zhang, Shiyu Chang +4
Prosody plays an important role in characterizing the style of a speaker or an emotion, but most non-parallel voice or emotion style transfer algorithms do not convert any prosody…
Cross-Modal Attention Consistency for Video-Audio Unsupervised Learning
Shaobo Min, Qi Dai, Hongtao Xie +3
Cross-modal correlation provides an inherent supervision for video unsupervised representation learning. Existing methods focus on distinguishing different video clips by visual an…
Curious Representation Learning for Embodied Intelligence
Yilun Du, Chuang Gan, Phillip Isola
Self-supervised representation learning has achieved remarkable success in recent years. By subverting the need for supervised labels, such approaches are able to utilize the numer…
TransCenter: Transformers with Dense Representations for Multiple-Object Tracking
Yihong Xu, Yutong Ban, Guillaume Delorme +3
Transformers have proven superior performance for a wide variety of tasks since they were introduced. In recent years, they have drawn attention from the vision community in tasks…
On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning
Ren Wang, Kaidi Xu, Sijia Liu +4
Model-agnostic meta-learning (MAML) has emerged as one of the most successful meta-learning techniques in few-shot learning. It enables us to learn a meta-initialization} of model…