777 citations · 1.9k across the 23 of their papers we have counts for
15 papers · 1 filter
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
Hao Zhang, Hongyang Li, Feng Li +8
With the recent significant advancements in large multi-modal models (LMMs), the importance of their grounding capability in visual chat is increasingly recognized. Despite recent…
Interfacing Foundation Models' Embeddings
Xueyan Zou, Linjie Li, Jianfeng Wang +10
Foundation models possess strong capabilities in reasoning and memorizing across modalities. To further unleash the power of foundation models, we present FIND, a generalized inter…
Visual In-Context Prompting
Feng Li, Qing Jiang, Hao Zhang +9
In-context prompting in large language models (LLMs) has become a prevalent approach to improve zero-shot capabilities, but this idea is less explored in the vision domain. Existin…
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
Shilong Liu, Hao Cheng, Haotian Liu +10
LLaVA-Plus is a general-purpose multimodal assistant that expands the capabilities of large multimodal models. It maintains a skill repository of pre-trained vision and vision-lang…
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
Jianwei Yang, Hao Zhang, Feng Li +3
We present Set-of-Mark (SoM), a new visual prompting method, to unleash the visual grounding abilities of large multimodal models (LMMs), such as GPT-4V. As illustrated in Fig. 1 (…
Semantic-SAM: Segment and Recognize Anything at Any Granularity
Feng Li, Hao Zhang, Peize Sun +6
In this paper, we introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity. Our model offers two key advanta…
detrex: Benchmarking Detection Transformers
Tianhe Ren, Shilong Liu, Feng Li +13
The DEtection TRansformer (DETR) algorithm has received considerable attention in the research community and is gradually emerging as a mainstream approach for object detection and…
A Strong and Reproducible Object Detector with Only Public Datasets
Tianhe Ren, Jianwei Yang, Shilong Liu +6
This work presents Focal-Stable-DINO, a strong and reproducible object detection model which achieves 64.6 AP on COCO val2017 and 64.8 AP on COCO test-dev using only 700M parameter…
Detection Transformer with Stable Matching
Shilong Liu, Tianhe Ren, Jiayu Chen +8
This paper is concerned with the matching stability problem across different decoder layers in DEtection TRansformers (DETR). We point out that the unstable matching in DETR is cau…
Segment Everything Everywhere All at Once
Xueyan Zou, Jianwei Yang, Hao Zhang +6
In this work, we present SEEM, a promptable and interactive model for segmenting everything everywhere all at once in an image, as shown in Fig.1. In SEEM, we propose a novel decod…
A Simple Framework for Open-Vocabulary Segmentation and Detection
Hao Zhang, Feng Li, Xueyan Zou +5
We present OpenSeeD, a simple Open-vocabulary Segmentation and Detection framework that jointly learns from different segmentation and detection datasets. To bridge the gap of voca…
MP-Former: Mask-Piloted Transformer for Image Segmentation
Hao Zhang, Feng Li, Huaizhe Xu +4
We present a mask-piloted Transformer which improves masked-attention in Mask2Former for image segmentation. The improvement is based on our observation that Mask2Former suffers fr…
Lite DETR : An Interleaved Multi-Scale Encoder for Efficient DETR
Feng Li, Ailing Zeng, Shilong Liu +4
Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance. Its success cannot be achieved without the re-introduction of multi-scale feature fusion in t…
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
Shilong Liu, Zhaoyang Zeng, Tianhe Ren +9
In this paper, we present an open-set object detector, called Grounding DINO, by marrying Transformer-based detector DINO with grounded pre-training, which can detect arbitrary obj…
Introducing Depth into Transformer-based 3D Object Detection
Hao Zhang, Hongyang Li, Ailing Zeng +4
In this paper, we present DAT, a Depth-Aware Transformer framework designed for camera-based 3D detection. Our model is based on observing two major issues in existing methods: lar…