777 citations · 1.9k across the 23 of their papers we have counts for
23 papers
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
Tianhe Ren, Shilong Liu, Ailing Zeng +14
We introduce Grounded SAM, which uses Grounding DINO as an open-set object detector to combine with the segment anything model (SAM). This integration enables the detection and seg…
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
Hao Zhang, Hongyang Li, Feng Li +8
With the recent significant advancements in large multi-modal models (LMMs), the importance of their grounding capability in visual chat is increasingly recognized. Despite recent…
Interfacing Foundation Models' Embeddings
Xueyan Zou, Linjie Li, Jianfeng Wang +10
Foundation models possess strong capabilities in reasoning and memorizing across modalities. To further unleash the power of foundation models, we present FIND, a generalized inter…
Visual In-Context Prompting
Feng Li, Qing Jiang, Hao Zhang +9
In-context prompting in large language models (LLMs) has become a prevalent approach to improve zero-shot capabilities, but this idea is less explored in the vision domain. Existin…
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
Shilong Liu, Hao Cheng, Haotian Liu +10
LLaVA-Plus is a general-purpose multimodal assistant that expands the capabilities of large multimodal models. It maintains a skill repository of pre-trained vision and vision-lang…
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
Jianwei Yang, Hao Zhang, Feng Li +3
We present Set-of-Mark (SoM), a new visual prompting method, to unleash the visual grounding abilities of large multimodal models (LMMs), such as GPT-4V. As illustrated in Fig. 1 (…
Semantic-SAM: Segment and Recognize Anything at Any Granularity
Feng Li, Hao Zhang, Peize Sun +6
In this paper, we introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity. Our model offers two key advanta…
detrex: Benchmarking Detection Transformers
Tianhe Ren, Shilong Liu, Feng Li +13
The DEtection TRansformer (DETR) algorithm has received considerable attention in the research community and is gradually emerging as a mainstream approach for object detection and…
A Strong and Reproducible Object Detector with Only Public Datasets
Tianhe Ren, Jianwei Yang, Shilong Liu +6
This work presents Focal-Stable-DINO, a strong and reproducible object detection model which achieves 64.6 AP on COCO val2017 and 64.8 AP on COCO test-dev using only 700M parameter…
Detection Transformer with Stable Matching
Shilong Liu, Tianhe Ren, Jiayu Chen +8
This paper is concerned with the matching stability problem across different decoder layers in DEtection TRansformers (DETR). We point out that the unstable matching in DETR is cau…
Segment Everything Everywhere All at Once
Xueyan Zou, Jianwei Yang, Hao Zhang +6
In this work, we present SEEM, a promptable and interactive model for segmenting everything everywhere all at once in an image, as shown in Fig.1. In SEEM, we propose a novel decod…
A Simple Framework for Open-Vocabulary Segmentation and Detection
Hao Zhang, Feng Li, Xueyan Zou +5
We present OpenSeeD, a simple Open-vocabulary Segmentation and Detection framework that jointly learns from different segmentation and detection datasets. To bridge the gap of voca…
MP-Former: Mask-Piloted Transformer for Image Segmentation
Hao Zhang, Feng Li, Huaizhe Xu +4
We present a mask-piloted Transformer which improves masked-attention in Mask2Former for image segmentation. The improvement is based on our observation that Mask2Former suffers fr…
Lite DETR : An Interleaved Multi-Scale Encoder for Efficient DETR
Feng Li, Ailing Zeng, Shilong Liu +4
Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance. Its success cannot be achieved without the re-introduction of multi-scale feature fusion in t…
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
Shilong Liu, Zhaoyang Zeng, Tianhe Ren +9
In this paper, we present an open-set object detector, called Grounding DINO, by marrying Transformer-based detector DINO with grounded pre-training, which can detect arbitrary obj…
Introducing Depth into Transformer-based 3D Object Detection
Hao Zhang, Hongyang Li, Ailing Zeng +4
In this paper, we present DAT, a Depth-Aware Transformer framework designed for camera-based 3D detection. Our model is based on observing two major issues in existing methods: lar…
DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding
Shilong Liu, Yaoyuan Liang, Feng Li +5
In this paper, we study the problem of visual grounding by considering both phrase extraction and grounding (PEG). In contrast to the previous phrase-known-at-test setting, PEG req…
A Unified Mutual Supervision Framework for Referring Expression Segmentation and Generation
Shijia Huang, Feng Li, Hao Zhang +3
Reference Expression Segmentation (RES) and Reference Expression Generation (REG) are mutually inverse tasks that can be naturally jointly trained. Though recent work has explored…
Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation
Feng Li, Hao Zhang, Huaizhe xu +4
In this paper we present Mask DINO, a unified object detection and segmentation framework. Mask DINO extends DINO (DETR with Improved Denoising Anchor Boxes) by adding a mask predi…
Vision-Language Intelligence: Tasks, Representation Learning, and Large Models
Feng Li, Hao Zhang, Yi-Fan Zhang +5
This paper presents a comprehensive survey of vision-language (VL) intelligence from the perspective of time. This survey is inspired by the remarkable progress in both computer vi…
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
Hao Zhang, Feng Li, Shilong Liu +5
We present DINO (\textbf{D}ETR with \textbf{I}mproved de\textbf{N}oising anch\textbf{O}r boxes), a state-of-the-art end-to-end object detector. % in this paper. DINO improves over…
DN-DETR: Accelerate DETR Training by Introducing Query DeNoising
Feng Li, Hao Zhang, Shilong Liu +3
We present in this paper a novel denoising training method to speedup DETR (DEtection TRansformer) training and offer a deepened understanding of the slow convergence issue of DETR…
DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR
Shilong Liu, Feng Li, Hao Zhang +5
We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR. This…