577 citations · 2.2k across the 64 of their papers we have counts for
16 papers · 1 filter
NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space
Jiawei Yao, Chuming Li, Keqiang Sun +4
Monocular 3D Semantic Scene Completion (SSC) has garnered significant attention in recent years due to its potential to predict complex semantics and geometry shapes from a single…
ImageBind-LLM: Multi-modality Instruction Tuning
Jiaming Han, Renrui Zhang, Wenqi Shao +14
We present ImageBind-LLM, a multi-modality instruction tuning method of large language models (LLMs) via ImageBind. Existing works mainly focus on language and image instruction tu…
Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following
Ziyu Guo, Renrui Zhang, Xiangyang Zhu +8
We introduce Point-Bind, a 3D multi-modality model aligning point clouds with 2D image, language, audio, and video. Guided by ImageBind, we construct a joint embedding space betwee…
Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigation
Jinyu Chen, Wenguan Wang, Si Liu +2
Audio-visual navigation is an audio-targeted wayfinding task where a robot agent is entailed to travel a never-before-seen 3D environment towards the sounding source. In this artic…
Denoising Diffusion Semantic Segmentation with Mask Prior Modeling
Zeqiang Lai, Yuchen Duan, Jifeng Dai +5
The evolution of semantic segmentation has long been dominated by learning more discriminative image representations for classifying each pixel. Despite the prominent advancements,…
FlowFormer: A Transformer Architecture and Its Masked Cost Volume Autoencoding for Optical Flow
Zhaoyang Huang, Xiaoyu Shi, Chao Zhang +6
This paper introduces a novel transformer-based network architecture, FlowFormer, along with the Masked Cost Volume AutoEncoding (MCVA) for pretraining it to tackle the problem of…
ADDP: Learning General Representations for Image Recognition and Generation with Alternating Denoising Diffusion Process
Changyao Tian, Chenxin Tao, Jifeng Dai +7
Image recognition and generation have long been developed independently of each other. With the recent trend towards general-purpose representation learning, the development of gen…
A Unified Conditional Framework for Diffusion-based Image Restoration
Yi Zhang, Xiaoyu Shi, Dasong Li +3
Diffusion Probabilistic Models (DPMs) have recently shown remarkable performance in image generation tasks, which are capable of generating highly realistic images. When adopting D…
Voxel2Hemodynamics: An End-to-end Deep Learning Method for Predicting Coronary Artery Hemodynamics
Ziyu Ni, Linda Wei, Lijian Xu +4
Local hemodynamic forces play an important role in determining the functional significance of coronary arterial stenosis and understanding the mechanism of coronary disease progres…
ReasonNet: End-to-End Driving with Temporal and Global Reasoning
Hao Shao, Letian Wang, Ruobing Chen +3
The large-scale deployment of autonomous vehicles is yet to come, and one of the major remaining challenges lies in urban dense traffic scenarios. In such cases, it remains challen…
Segmentation and Vascular Vectorization for Coronary Artery by Geometry-based Cascaded Neural Network
Xiaoyu Yang, Lijian Xu, Simon Yu +3
Segmentation of the coronary artery is an important task for the quantitative analysis of coronary computed tomography angiography (CCTA) images and is being stimulated by the fiel…
KBNet: Kernel Basis Network for Image Restoration
Yi Zhang, Dasong Li, Xiaoyu Shi +5
How to aggregate spatial information plays an essential role in learning-based image restoration. Most existing CNN-based networks adopt static convolutional kernels to encode spat…
FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation
Xiaoyu Shi, Zhaoyang Huang, Dasong Li +6
FlowFormer introduces a transformer architecture into optical flow estimation and achieves state-of-the-art performance. The core component of FlowFormer is the transformer-based c…
GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding
Jihao Liu, Tai Wang, Boxiao Liu +3
Multi-view camera-based 3D detection is a challenging problem in computer vision. Recent works leverage a pretrained LiDAR detection model to transfer knowledge to a camera-based s…
BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation
Yijin Li, Zhaoyang Huang, Shuo Chen +5
Event cameras provide high temporal precision, low data rates, and high dynamic range visual perception, which are well-suited for optical flow estimation. While data-driven optica…
PATS: Patch Area Transportation with Subdivision for Local Feature Matching
Junjie Ni, Yijin Li, Zhaoyang Huang +4
Local feature matching aims at establishing sparse correspondences between a pair of images. Recently, detector-free methods present generally better performance but are not satisf…