11 citations · 17 across the 14 of their papers we have counts for
17 papers
Learning Representations to Guide Pixel Diffusion Transformers
Zihao Pan, Mengting Chen, Cheng Shi +6
Visual representations are largely developed for understanding, yet what makes them useful for generation remains unclear. We study this question in pixel diffusion transformers an…
Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models
Aiwei Liu, Cheng Shi, Chuhan Wu +44
Scaling Large Language Models (LLMs) has been driven mainly by enlarging the Transformer backbone, but for an already-strong model this requires another round of costly pretraining…
WeaveTime: Stream from Earlier Frames into Emergent Memory in VideoLLMs
Yulin Zhang, Cheng Shi, Sibei Yang
Recent advances in Multimodal Large Language Models have greatly improved visual understanding and reasoning, yet their quadratic attention and offline training protocols make them…
Vision Transformers Need More Than Registers
Cheng Shi, Yizhou Yu, Sibei Yang
Vision Transformers (ViTs), when pre-trained on large-scale data, provide general-purpose representations for diverse downstream tasks. However, artifacts in ViTs are widely observ…
Eyes Wide Open: Ego Proactive Video-LLM for Streaming Video
Yulin Zhang, Cheng Shi, Yang Wang +1
Envision an AI capable of functioning in human-like settings, moving beyond mere observation to actively understand, anticipate, and proactively respond to unfolding events. Toward…
Vision Function Layer in Multimodal LLMs
Cheng Shi, Yizhou Yu, Sibei Yang
This study identifies that visual-related functional decoding is distributed across different decoder layers in Multimodal Large Language Models (MLLMs). Typically, each function,…
Sim-DETR: Unlock DETR for Temporal Sentence Grounding
Jiajin Tang, Zhengxuan Wei, Yuchen Zhu +4
Temporal sentence grounding aims to identify exact moments in a video that correspond to a given textual query, typically addressed with detection transformer (DETR) solutions. How…
Rethinking Query-based Transformer for Continual Image Segmentation
Yuchen Zhu, Cheng Shi, Dingyou Wang +5
Class-incremental/Continual image segmentation (CIS) aims to train an image segmenter in stages, where the set of available categories differs at each stage. To leverage the built-…
Part2Object: Hierarchical Unsupervised 3D Instance Segmentation
Cheng Shi, Yulin Zhang, Bin Yang +3
Unsupervised 3D instance segmentation aims to segment objects from a 3D point cloud without any annotations. Existing methods face the challenge of either too loose or too tight cl…
Plain-Det: A Plain Multi-Dataset Object Detector
Cheng Shi, Yuchen Zhu, Sibei Yang
Recent advancements in large-scale foundational models have sparked widespread interest in training highly proficient large vision models. A common consensus revolves around the ne…
The devil is in the object boundary: towards annotation-free instance segmentation using Foundation Models
Cheng Shi, Sibei Yang
Foundation models, pre-trained on a large amount of data have demonstrated impressive zero-shot capabilities in various downstream tasks. However, in object detection and instance…
Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator
Hanzhuo Huang, Yufan Feng, Cheng Shi +3
Text-to-video is a rapidly growing research area that aims to generate a semantic, identical, and temporal coherence sequence of frames that accurately align with the input text pr…
LoGoPrompt: Synthetic Text Images Can Be Good Visual Prompts for Vision-Language Models
Cheng Shi, Sibei Yang
Prompt engineering is a powerful tool used to enhance the performance of pre-trained models on downstream tasks. For example, providing the prompt "Let's think step by step" improv…
EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignment
Cheng Shi, Sibei Yang
Vision-language models such as CLIP have boosted the performance of open-vocabulary object detection, where the detector is trained on base categories but required to detect novel…
Spatial and Visual Perspective-Taking via View Rotation and Relation Reasoning for Embodied Reference Understanding
Cheng Shi, Sibei Yang
Embodied Reference Understanding studies the reference understanding in an embodied fashion, where a receiver is required to locate a target object referred to by both language and…
Contrastive Grouping with Transformer for Referring Image Segmentation
Jiajin Tang, Ge Zheng, Cheng Shi +1
Referring image segmentation aims to segment the target referent in an image conditioning on a natural language expression. Existing one-stage methods employ per-pixel classificati…
DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance
Longwen Zhang, Qiwei Qiu, Hongyang Lin +7
Emerging Metaverse applications demand accessible, accurate, and easy-to-use tools for 3D digital human creations in order to depict different cultures and societies as if in the p…