284 citations · 995 across the 32 of their papers we have counts for
17 papers · 1 filter
Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation
Yitong Li, Junsong Chen, Haopeng Li +6
Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost. Although many acceleration methods have been proposed, a ce…
Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning
Changye Li, Meng Lu, Yi Wu +1
While recent vision-language models (VLMs) demonstrate strong multimodal understanding, they remain limited in spatial reasoning tasks that require active evidence acquisition and…
Grounded 3D-Aware Spatial Vision-Language Modeling
An-Chieh Cheng, Yang Fu, Yatai Ji +12
We present GR3D, a spatial vision language model equipped with three complementary grounding capabilities--explicit 2D grounding, implicit 2D grounding, and monocular 3D grounding-…
EGM: Efficient Visual Grounding Language Models
Guanqi Zhan, Changye Li, Zhijian Liu +4
Visual grounding is an essential capability of Visual Language Models (VLMs) to understand the real physical world. Previous state-of-the-art grounding visual language models usual…
FoundationMotion: Auto-Labeling and Reasoning about Spatial Movement in Videos
Yulu Gan, Ligeng Zhu, Dandan Shan +8
Motion understanding is fundamental to physical reasoning, enabling models to infer dynamics and predict future states. However, state-of-the-art models still struggle on recent mo…
OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM
Hanrong Ye, Chao-Han Huck Yang, Arushi Goel +29
Advancing machine intelligence requires developing the ability to perceive across multiple modalities, much as humans sense the world. We introduce OmniVinci, an initiative to buil…
DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder
Junyu Chen, Wenkun He, Yuchao Gu +12
We introduce DC-VideoGen, a post-training acceleration framework for efficient video generation. DC-VideoGen can be applied to any pre-trained video diffusion model, improving effi…
DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space
Wenkun He, Yuchao Gu, Junyu Chen +11
Existing text-to-image diffusion models excel at generating high-quality images, but face significant efficiency challenges when scaled to high resolutions, like 4K image generatio…
Scaling RL to Long Videos
Yukang Chen, Wei Huang, Baifeng Shi +11
We introduce a full-stack framework that scales up reasoning in vision-language models (VLMs) to long videos, leveraging reinforcement learning. We address the unique challenges of…
SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer
Enze Xie, Junsong Chen, Yuyang Zhao +11
This paper presents SANA-1.5, a linear Diffusion Transformer for efficient scaling in text-to-image generation. Building upon SANA-1.0, we introduce three key innovations: (1) Effi…
NVILA: Efficient Frontier Visual Language Models
Zhijian Liu, Ligeng Zhu, Baifeng Shi +24
Visual language models (VLMs) have made significant advances in accuracy in recent years. However, their efficiency has received much less attention. This paper introduces NVILA, a…
SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers
Enze Xie, Junsong Chen, Junyu Chen +8
We introduce Sana, a text-to-image framework that can efficiently generate images up to 40964096 resolution. Sana can synthesize high-resolution, high-quality images with s…
VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation
Yecheng Wu, Zhuoyang Zhang, Junyu Chen +9
VILA-U is a Unified foundation model that integrates Video, Image, Language understanding and generation. Traditional visual language models (VLMs) use separate modules for underst…
LongVILA: Scaling Long-Context Visual Language Models for Long Videos
Yukang Chen, Fuzhao Xue, Dacheng Li +15
Long-context capability is critical for multi-modal foundation models, especially for long video understanding. We introduce LongVILA, a full-stack solution for long-context visual…
VILA: VILA Augmented VILA
Yunhao Fang, Ligeng Zhu, Yao Lu +7
While visual language model architectures and training infrastructures advance rapidly, data curation remains under-explored where quantity and quality become a bottleneck. Existin…
On-Device Training Under 256KB Memory
Ji Lin, Ligeng Zhu, Wei-Ming Chen +3
On-device training enables the model to adapt to new data collected from the sensors by fine-tuning a pre-trained model. Users can benefit from customized AI models without having…
TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning
Han Cai, Chuang Gan, Ligeng Zhu +1
On-device learning enables edge devices to continually adapt the AI models to new data, which requires a small memory footprint to fit the tight memory constraint of edge devices.…