577 citations · 2.2k across the 64 of their papers we have counts for
66 papers
NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space
Jiawei Yao, Chuming Li, Keqiang Sun +4
Monocular 3D Semantic Scene Completion (SSC) has garnered significant attention in recent years due to its potential to predict complex semantics and geometry shapes from a single…
ImageBind-LLM: Multi-modality Instruction Tuning
Jiaming Han, Renrui Zhang, Wenqi Shao +14
We present ImageBind-LLM, a multi-modality instruction tuning method of large language models (LLMs) via ImageBind. Existing works mainly focus on language and image instruction tu…
Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following
Ziyu Guo, Renrui Zhang, Xiangyang Zhu +8
We introduce Point-Bind, a 3D multi-modality model aligning point clouds with 2D image, language, audio, and video. Guided by ImageBind, we construct a joint embedding space betwee…
Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigation
Jinyu Chen, Wenguan Wang, Si Liu +2
Audio-visual navigation is an audio-targeted wayfinding task where a robot agent is entailed to travel a never-before-seen 3D environment towards the sounding source. In this artic…
Denoising Diffusion Semantic Segmentation with Mask Prior Modeling
Zeqiang Lai, Yuchen Duan, Jifeng Dai +5
The evolution of semantic segmentation has long been dominated by learning more discriminative image representations for classifying each pixel. Despite the prominent advancements,…
FlowFormer: A Transformer Architecture and Its Masked Cost Volume Autoencoding for Optical Flow
Zhaoyang Huang, Xiaoyu Shi, Chao Zhang +6
This paper introduces a novel transformer-based network architecture, FlowFormer, along with the Masked Cost Volume AutoEncoding (MCVA) for pretraining it to tackle the problem of…
ADDP: Learning General Representations for Image Recognition and Generation with Alternating Denoising Diffusion Process
Changyao Tian, Chenxin Tao, Jifeng Dai +7
Image recognition and generation have long been developed independently of each other. With the recent trend towards general-purpose representation learning, the development of gen…
A Unified Conditional Framework for Diffusion-based Image Restoration
Yi Zhang, Xiaoyu Shi, Dasong Li +3
Diffusion Probabilistic Models (DPMs) have recently shown remarkable performance in image generation tasks, which are capable of generating highly realistic images. When adopting D…
Voxel2Hemodynamics: An End-to-end Deep Learning Method for Predicting Coronary Artery Hemodynamics
Ziyu Ni, Linda Wei, Lijian Xu +4
Local hemodynamic forces play an important role in determining the functional significance of coronary arterial stenosis and understanding the mechanism of coronary disease progres…
ReasonNet: End-to-End Driving with Temporal and Global Reasoning
Hao Shao, Letian Wang, Ruobing Chen +3
The large-scale deployment of autonomous vehicles is yet to come, and one of the major remaining challenges lies in urban dense traffic scenarios. In such cases, it remains challen…
Segmentation and Vascular Vectorization for Coronary Artery by Geometry-based Cascaded Neural Network
Xiaoyu Yang, Lijian Xu, Simon Yu +3
Segmentation of the coronary artery is an important task for the quantitative analysis of coronary computed tomography angiography (CCTA) images and is being stimulated by the fiel…
KBNet: Kernel Basis Network for Image Restoration
Yi Zhang, Dasong Li, Xiaoyu Shi +5
How to aggregate spatial information plays an essential role in learning-based image restoration. Most existing CNN-based networks adopt static convolutional kernels to encode spat…
FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation
Xiaoyu Shi, Zhaoyang Huang, Dasong Li +6
FlowFormer introduces a transformer architecture into optical flow estimation and achieves state-of-the-art performance. The core component of FlowFormer is the transformer-based c…
GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding
Jihao Liu, Tai Wang, Boxiao Liu +3
Multi-view camera-based 3D detection is a challenging problem in computer vision. Recent works leverage a pretrained LiDAR detection model to transfer knowledge to a camera-based s…
BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation
Yijin Li, Zhaoyang Huang, Shuo Chen +5
Event cameras provide high temporal precision, low data rates, and high dynamic range visual perception, which are well-suited for optical flow estimation. While data-driven optica…
PATS: Patch Area Transportation with Subdivision for Local Feature Matching
Junjie Ni, Yijin Li, Zhaoyang Huang +4
Local feature matching aims at establishing sparse correspondences between a pair of images. Recently, detector-free methods present generally better performance but are not satisf…
ConQueR: Query Contrast Voxel-DETR for 3D Object Detection
Benjin Zhu, Zhe Wang, Shaoshuai Shi +3
Although DETR-based 3D detectors can simplify the detection pipeline and achieve direct sparse predictions, their performance still lags behind dense detectors with post-processing…
Teach-DETR: Better Training DETR with Teachers
Linjiang Huang, Kaixin Lu, Guanglu Song +4
In this paper, we present a novel training scheme, namely Teach-DETR, to learn better DETR-based detectors from versatile teacher detectors. We show that the predicted boxes from t…
Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks
Hao Li, Jinguo Zhu, Xiaohu Jiang +8
Despite the remarkable success of foundation models, their task-specific fine-tuning paradigm makes them inconsistent with the goal of general perception modeling. The key to elimi…
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Wenhai Wang, Jifeng Dai, Zhe Chen +9
Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early sta…
NeuralMarker: A Framework for Learning General Marker Correspondence
Zhaoyang Huang, Xiaokun Pan, Weihong Pan +5
We tackle the problem of estimating correspondences from a general marker, such as a movie poster, to an image that captures such a marker. Conventionally, this problem is addresse…
Magnetic Resonance Fingerprinting with compressed sensing and distance metric learning
Zhe Wang, Hongsheng Li, Qinwei Zhang +2
Magnetic Resonance Fingerprinting (MRF) is a novel technique that simultaneously estimates multiple tissue-related parameters, such as the longitudinal relaxation time T1, the tran…
Learning Degradation Representations for Image Deblurring
Dasong Li, Yi Zhang, Ka Chun Cheung +3
In various learning-based image restoration tasks, such as image denoising and image super-resolution, the degradation representations were widely used to model the degradation pro…
Frozen CLIP Models are Efficient Video Learners
Ziyi Lin, Shijie Geng, Renrui Zhang +6
Video recognition has been dominated by the end-to-end learning paradigm -- first initializing a video recognition model with weights of a pretrained image model and then conductin…