11 papers · 1 filter
Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation
Weiming Zhuang, Jiabo Huang, Jingtao Li +4
Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between t…
On the Limits of Token Reduction for Efficient Unified Vision Language Training
Siyi Chen, Weiming Zhuang, Jingtao Li +1
Unified vision-language models (VLMs) integrate visual understanding and visual generation within a single autoregressive backbone, but their joint training is computationally expe…
HyperVision: A Channel-Adaptive Ground-Based Hyperspectral Vision Pre-trained Backbone
Guanyiman Fu, Jingtao Li, Zihang Cheng +8
While hyperspectral imaging provides rich spatial-spectral information across hundreds of narrow wavelength bands for precise material identification, ground-based hyperspectral pr…
VibeToken: Scaling 1D Image Tokenizers and Autoregressive Models for Dynamic Resolution Generations
Maitreya Patel, Jingtao Li, Weiming Zhuang +2
We introduce an efficient, resolution-agnostic autoregressive (AR) image synthesis approach that generalizes to arbitrary resolutions and aspect ratios, narrowing the gap to diffus…
UniCompress: Token Compression for Unified Vision-Language Understanding and Generation
Ziyao Wang, Chen Chen, Jingtao Li +4
Unified models aim to support both understanding and generation by encoding images into discrete tokens and processing them alongside text within a single autoregressive framework.…
UNIFORM: Unifying Knowledge from Large-scale and Diverse Pre-trained Models
Yimu Wang, Weiming Zhuang, Chen Chen +3
In the era of deep learning, the increasing number of pre-trained models available online presents a wealth of knowledge. These models, developed with diverse architectures and tra…
Training-Free Layout-to-Image Generation with Marginal Attention Constraints
Huancheng Chen, Jingtao Li, Weiming Zhuang +2
Recently, many text-to-image diffusion models have excelled at generating high-resolution images from text but struggle with precise control over spatial composition and object cou…
Replay-Free Continual Low-Rank Adaptation with Dynamic Memory
Huancheng Chen, Jingtao Li, Weiming Zhuang +2
We revisit continual learning~(CL), which enables pre-trained vision transformers (ViTs) to sequentially fine-tune on new downstream tasks over time. However, as the scale of these…
COALA: A Practical and Vision-Centric Federated Learning Platform
Weiming Zhuang, Jian Xu, Chen Chen +2
We present COALA, a vision-centric Federated Learning (FL) platform, and a suite of benchmarks for practical FL scenarios, which we categorize into three levels: task, data, and mo…
Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget
Vikash Sehwag, Xianghao Kong, Jingtao Li +2
As scaling laws in generative AI push performance, they also simultaneously concentrate the development of these models among actors with large computational resources. With a focu…
Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization
Yuhang Li, Xin Dong, Chen Chen +4
Synthetic image data generation represents a promising avenue for training deep learning models, particularly in the realm of transfer learning, where obtaining real images within…