69 citations · 129 across the 27 of their papers we have counts for
30 papers
EVA-Cap: Optimizing Audiovisual Video Captioning via Event-Centric Alignment
Jiapeng Shi, Qiuxia Hou, Jiaqi Leng +5
Recent advances in Omni Language Models (OLMs) provide a unified solution for audiovisual video captioning. Nevertheless, existing approaches predominantly treat captioning as an u…
Native Video-Action Pretraining for Generalizable Robot Control
Qihang Zhang, Lin Li, Luyao Zhang +26
The advent of video-action models offers a promising path for robot control. Nevertheless, we argue that repurposing video generative models designed for digital content creation i…
ThinkingVLA: Interleaved Vision and Language Reasoning for Robotic Manipulation
Tianyi Lu, Hui Zhang, Zijie Diao +8
Most Vision-Language-Action (VLA) models map observations directly to actions without explicit reasoning, limiting their capacity for reasoning-intensive long-horizon tasks. To add…
RepWAM: World Action Modeling with Representation Visual-Action Tokenizers
Junke Wang, Qihang Zhang, Shuai Yang +5
This work presents RepWAM, a representation-centric world action model (WAM) built on representation visual-action tokenizers. Existing WAMs typically inherit reconstruction-orient…
ARM: An AutoRegressive Large Multimodal Model with Unified Discrete Representations
Junke Wang, Xiao Wang, Jiacheng Pan +16
This paper introduces ARM, a discrete representation-based AutoRegressive Model that unifies image understanding, generation, and editing within a next-token prediction framework.…
IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder
Yitong Chen, Zijie Diao, Junke Wang +5
Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spac…
OmniGen-AR: AutoRegressive Any-to-Image Generation
Junke Wang, Xun Wang, Qiushan Guo +4
Autoregressive (AR) models have demonstrated strong potential in visual generation, offering superior performance with simple architectures and optimization objectives. However, ex…
DisCo: World Models with Discrete Camera Motion Control
Hongrui Huang, Junke Wang, Quanhao Li +2
Controllable video world models target interactive world exploration, where models must faithfully execute explicit action commands while preserving visual quality and temporal coh…
FluxMem: Adaptive Hierarchical Memory for Streaming Video Understanding
Yiweng Xie, Bo He, Junke Wang +3
This paper presents FluxMem, a training-free framework for efficient streaming video understanding. FluxMem adaptively compresses redundant visual memory through a hierarchical, tw…
VideoLoom: A Video Large Language Model for Joint Spatial-Temporal Understanding
Jiapeng Shi, Junke Wang, Zuyao You +2
This paper presents VideoLoom, a unified Video Large Language Model (Video LLM) for joint spatial-temporal understanding. To facilitate the development of fine-grained spatial and…
DeRA: Decoupled Representation Alignment for Video Tokenization
Pengbo Guo, Junke Wang, Zhen Xing +4
This paper presents DeRA, a novel 1D video tokenizer that decouples the spatial-temporal representation learning in video tokenization to achieve better training efficiency and per…
TempoMaster: Efficient Long Video Generation via Next-Frame-Rate Prediction
Yukuo Ma, Cong Liu, Junke Wang +5
We present TempoMaster, a novel framework that formulates long video generation as next-frame-rate prediction. Specifically, we first generate a low-frame-rate clip that serves as…
Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis
Peng Zheng, Junke Wang, Yi Chang +3
Recent advances in large language models (LLMs) have spurred interests in encoding images as discrete tokens and leveraging autoregressive (AR) frameworks for visual generation. Ho…
Perception Encoder: The best visual embeddings are not at the output of the network
Daniel Bolya, Po-Yao Huang, Peize Sun +15
We introduce Perception Encoder (PE), a state-of-the-art vision encoder for image and video understanding trained via simple vision-language learning. Traditionally, vision encoder…
SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RL
Junke Wang, Zhi Tian, Xun Wang +4
This work presents SimpleAR, a vanilla autoregressive visual generation framework without complex architecure modifications. Through careful exploration of training and inference o…
Pix2Cap-COCO: Advancing Visual Comprehension via Pixel-Level Captioning
Zuyao You, Junke Wang, Lingyu Kong +2
We present Pix2Cap-COCO, the first panoptic pixel-level caption dataset designed to advance fine-grained visual understanding. To achieve this, we carefully design an automated ann…
OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation
Junke Wang, Yi Jiang, Zehuan Yuan +3
Tokenizer, serving as a translator to map the intricate visual data into a compact latent space, lies at the core of visual generative models. Based on the finding that existing to…
OmniVid: A Generative Framework for Universal Video Understanding
Junke Wang, Dongdong Chen, Chong Luo +4
The core of video understanding tasks, such as recognition, captioning, and tracking, is to automatically detect objects or actions in a video and analyze their temporal evolution.…
MouSi: Poly-Visual-Expert Vision-Language Models
Xiaoran Fan, Tao Ji, Changhao Jiang +21
Current large vision-language models (VLMs) often encounter challenges such as insufficient capabilities of a single visual component and excessively long visual tokens. These issu…
To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuning
Junke Wang, Lingchen Meng, Zejia Weng +3
Existing visual instruction tuning methods typically prompt large language models with textual descriptions to generate instruction-following data. Despite the promising performanc…
ChatVideo: A Tracklet-centric Multimodal and Versatile Video Understanding System
Junke Wang, Dongdong Chen, Chong Luo +4
Existing deep video models are limited by specific tasks, fixed input-output spaces, and poor generalization capabilities, making it difficult to deploy them in real-world scenario…
OmniTracker: Unifying Object Tracking by Tracking-with-Detection
Junke Wang, Zuxuan Wu, Dongdong Chen +4
Visual Object Tracking (VOT) aims to estimate the positions of target objects in a video sequence, which is an important vision task with various real-world applications. Depending…
Look Before You Match: Instance Understanding Matters in Video Object Segmentation
Junke Wang, Dongdong Chen, Zuxuan Wu +7
Exploring dense matching between the current frame and past frames for long-range context modeling, memory-based methods have demonstrated impressive results in video object segmen…
Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection
Junke Wang, Zhenxin Li, Chao Zhang +4
Online media data, in the forms of images and videos, are becoming mainstream communication channels. However, recent advances in deep learning, particularly deep generative models…