46 citations · 270 across the 24 of their papers we have counts for
24 papers
From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models
Rongjie Li, Songyang Zhang, Dahua Lin +2
Scene graph generation (SGG) aims to parse a visual scene into an intermediate graph representation for downstream reasoning tasks. Despite recent advancements, existing methods st…
SGTR+: End-to-end Scene Graph Generation with Transformer
Rongjie Li, Songyang Zhang, Xuming He
Scene Graph Generation (SGG) remains a challenging visual understanding task due to its compositional property. Most previous works adopt a bottom-up, two-stage or point-based, one…
BotChat: Evaluating LLMs' Capabilities of Having Multi-Turn Dialogues
Haodong Duan, Jueqi Wei, Chonghua Wang +5
Interacting with human via high-quality multi-turn dialogues is a key feature of large language models (LLMs). However, human-based evaluation of such capability involves intensive…
LawBench: Benchmarking Legal Knowledge of Large Language Models
Zhiwei Fei, Xiaoyu Shen, Dawei Zhu +6
Large language models (LLMs) have demonstrated strong capabilities in various aspects. However, when applying them to the highly specialized, safe-critical legal domain, it is uncl…
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition
Pan Zhang, Xiaoyi Dong, Bin Wang +18
We propose InternLM-XComposer, a vision-language large model that enables advanced image-text comprehension and composition. The innovative nature of our model is highlighted by th…
Improving Pixel-based MIM by Reducing Wasted Modeling Capability
Yuan Liu, Songyang Zhang, Jiacheng Chen +3
There has been significant progress in Masked Image Modeling (MIM). Existing MIM methods can be broadly categorized into two groups based on the reconstruction target: pixel-based…
MMBench: Is Your Multi-modal Model an All-around Player?
Yuan Liu, Haodong Duan, Yuanhan Zhang +9
Large vision-language models (VLMs) have recently achieved remarkable progress, exhibiting impressive multimodal perception and reasoning abilities. However, effectively evaluating…
RIFormer: Keep Your Vision Backbone Effective While Removing Token Mixer
Jiahao Wang, Songyang Zhang, Yong Liu +6
This paper studies how to keep a vision backbone effective while removing token mixers in its basic building blocks. Token mixers, as self-attention for vision transformers (ViTs),…
PixMIM: Rethinking Pixel Reconstruction in Masked Image Modeling
Yuan Liu, Songyang Zhang, Jiacheng Chen +2
Masked Image Modeling (MIM) has achieved promising progress with the advent of Masked Autoencoders (MAE) and BEiT. However, subsequent works have complicated the framework with new…
Dynamic Grained Encoder for Vision Transformers
Lin Song, Songyang Zhang, Songtao Liu +5
Transformers, the de-facto standard for language modeling, have been recently applied for vision tasks. This paper introduces sparse queries for vision transformers to exploit the…
Learning Semantic Correspondence with Sparse Annotations
Shuaiyi Huang, Luyu Yang, Bo He +3
Finding dense semantic correspondence is a fundamental problem in computer vision, which remains challenging in complex scenes due to background clutter, extreme intra-class variat…
Budget-aware Few-shot Learning via Graph Convolutional Network
Shipeng Yan, Songyang Zhang, Xuming He
This paper tackles the problem of few-shot learning, which aims to learn new visual concepts from a few examples. A common problem setting in few-shot classification assumes random…
SGTR: End-to-end Scene Graph Generation with Transformer
Rongjie Li, Songyang Zhang, Xuming He
Scene Graph Generation (SGG) remains a challenging visual understanding task due to its compositional property. Most previous works adopt a bottom-up two-stage or a point-based one…
An EM Framework for Online Incremental Learning of Semantic Segmentation
Shipeng Yan, Jiale Zhou, Jiangwei Xie +2
Incremental learning of semantic segmentation has emerged as a promising strategy for visual scene interpretation in the open- world setting. However, it remains challenging to acq…
Workshop on Autonomous Driving at CVPR 2021: Technical Report for Streaming Perception Challenge
Songyang Zhang, Lin Song, Songtao Liu +4
In this report, we introduce our real-time 2D object detection system for the realistic autonomous driving scenario. Our detector is built on a newly designed YOLO model, called YO…
Learning Implicit Temporal Alignment for Few-shot Video Classification
Songyang Zhang, Jiale Zhou, Xuming He
Few-shot video classification aims to learn new video categories with only a few labeled examples, alleviating the burden of costly annotation in real-world applications. However,…
Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation
Rongjie Li, Songyang Zhang, Bo Wan +1
Scene graph generation is an important visual understanding task with a broad range of vision applications. Despite recent tremendous progress, it remains challenging due to the in…
Distribution Alignment: A Unified Framework for Long-tail Visual Recognition
Songyang Zhang, Zeming Li, Shipeng Yan +2
Despite the recent success of deep neural networks, it remains challenging to effectively model the long-tail class distribution in visual recognition tasks. To address this proble…
An Efficient Hypergraph Approach to Robust Point Cloud Resampling
Qinwen Deng, Songyang Zhang, Zhi Ding
Efficient processing and feature extraction of largescale point clouds are important in related computer vision and cyber-physical systems. This work investigates point cloud resam…
Transformer with Bidirectional Decoder for Speech Recognition
Xi Chen, Songyang Zhang, Dandan Song +2
Attention-based models have made tremendous progress on end-to-end automatic speech recognition(ASR) recently. However, the conventional transformer-based approaches usually genera…
Part-aware Prototype Network for Few-shot Semantic Segmentation
Yongfei Liu, Xiangyi Zhang, Songyang Zhang +1
Few-shot semantic segmentation aims to learn to segment new object classes with only a few annotated examples, which has a wide range of real-world applications. Most existing meth…
Dynamic Context Correspondence Network for Semantic Alignment
Shuaiyi Huang, Qiuyue Wang, Songyang Zhang +2
Establishing semantic correspondence is a core problem in computer vision and remains challenging due to large intra-class variations and lack of annotated data. In this paper, we…
LatentGNN: Learning Efficient Non-local Relations for Visual Recognition
Songyang Zhang, Shipeng Yan, Xuming He
Capturing long-range dependencies in feature representations is crucial for many visual recognition tasks. Despite recent successes of deep convolutional networks, it remains chall…
Generalization Tower Network: A Novel Deep Neural Network Architecture for Multi-Task Learning
Yuhang Song, Main Xu, Songyang Zhang +1
Deep learning (DL) advances state-of-the-art reinforcement learning (RL), by incorporating deep neural networks in learning representations from the input to RL. However, the conve…