19 papers
How Far Can Coding Agents Go in Visual Generation?
Hengguang Zhou, Yuanhao Ban, Jiaqi Feng +3
Modern image generation largely relies on specialized generative models that leave the process of image construction implicit in learned parameters. We explore an alternative: gene…
SlackDrive: Reclaiming Runtime Slack for Adaptive Driving Inference
Xiaohuan Pei, Hengguang Zhou, Yuanhao Ban +7
Driving world-action models improve planning by coupling multimodal reasoning with future prediction, but their growing inference cost increasingly conflicts with the real-time lat…
Stream4D: 4D-Consistency for Streaming Autoregressive Diffusion Video Models
Yuanhao Ban, Jiaqi Feng, Hengguang Zhou +3
Streaming autoregressive diffusion models enable real-time, long-horizon video generation, but their training objectives optimize local frame prediction rather than the geometry an…
Self-Evolving Visual Questioner
Yijun Liang, Hengguang Zhou, Ming Li +3
Vision-language models (VLMs) are typically trained as passive answerers, while their ability to actively ask diverse, non-trivial, visual-centric and grounded questions remains un…
One-Forcing: Towards Stable One-Step Autoregressive Video Generation
Jiaqi Feng, Justin Cui, Yuanhao Ban +1
Recent advances in autoregressive diffusion-based video generation have substantially improved the quality of real-time video synthesis. However, most existing methods still requir…
AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment
Kuei-Chun Kao, Daixuan Huo, Yuanhao Ban +1
Aligning Text-to-Image (T2I) generation models with human preferences increasingly relies on image reward models that score or rank generated images according to prompt alignment a…
LoL: Longer than Longer, Scaling Video Generation to Hour
Justin Cui, Jie Wu, Ming Li +6
Recent research in long-form video generation has shifted from bidirectional to autoregressive models, yet these methods commonly suffer from error accumulation and a loss of long-…
When Distance Distracts: Representation Distance Bias in BT-Loss for Reward Models
Tong Xie, Andrew Bai, Yuanhao Ban +3
Reward models are central to Large Language Model (LLM) alignment within the framework of RLHF. The standard objective used in reward modeling is the Bradley-Terry (BT) loss, which…
Uncertainty-Guided Selective Adaptation Enables Cross-Platform Predictive Fluorescence Microscopy
Kai-Wen K. Yang, Andrew Bai, Alexandra Bermudez +7
Deep learning is transforming microscopy, yet models often fail when applied to images from new instruments or acquisition settings. Conventional adversarial domain adaptation (ADD…
Self-Forcing++: Towards Minute-Scale High-Quality Video Generation
Justin Cui, Jie Wu, Ming Li +6
Diffusion models have revolutionized image and video generation, achieving unprecedented visual quality. However, their reliance on transformer architectures incurs prohibitively h…
IRIS: Intrinsic Reward Image Synthesis
Yihang Chen, Yuanhao Ban, Yunqi Hong +1
Despite the success of Reinforcement Learning from Human Feedback (RLHF) in language reasoning, its application to autoregressive Text-to-Image (T2I) generation is often constraine…
Concepts or Skills? Rethinking Instruction Selection for Multi-modal Models
Andrew Bai, Justin Cui, Ruochen Wang +1
Vision-language instruction tuning achieves two main purposes: learning visual concepts and learning visual skills. In this paper, we found that vision-language benchmarks fall int…
Unlabeled Data Improves Fine-Grained Image Zero-shot Classification with Multimodal LLMs
Yunqi Hong, Sohyun An, Andrew Bai +2
Despite Multimodal Large Language Models (MLLMs) showing promising results on general zero-shot image classification tasks, fine-grained image classification remains challenging. I…
On the Loss of Context-awareness in General Instruction Fine-tuning
Yihan Wang, Andrew Bai, Nanyun Peng +1
Pre-trained Large Language Models (LLMs) require post-training methods such as supervised fine-tuning (SFT) on instruction-response pairs to enable instruction following. However,…
CLUE: Concept-Level Uncertainty Estimation for Large Language Models
Yu-Hsiang Wang, Andrew Bai, Che-Ping Tsai +1
Large Language Models (LLMs) have demonstrated remarkable proficiency in various natural language generation (NLG) tasks. Previous studies suggest that LLMs' generation process inv…
Embedding Space Selection for Detecting Memorization and Fingerprinting in Generative Models
Jack He, Jianxing Zhao, Andrew Bai +1
In the rapidly evolving landscape of artificial intelligence, generative models such as Generative Adversarial Networks (GANs) and Diffusion Models have become cornerstone technolo…
Defending LLMs against Jailbreaking Attacks via Backtranslation
Yihan Wang, Zhouxing Shi, Andrew Bai +1
Although many large language models (LLMs) have been trained to refuse harmful requests, they are still vulnerable to jailbreaking attacks which rewrite the original prompt to conc…
An Efficient Rehearsal Scheme for Catastrophic Forgetting Mitigation during Multi-stage Fine-tuning
Andrew Bai, Chih-Kuan Yeh, Cho-Jui Hsieh +1
Incrementally fine-tuning foundational models on new tasks or domains is now the de facto approach in NLP. A known pitfall of this approach is the \emph{catastrophic forgetting} of…
Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation
Tong Xie, Haoyu Li, Andrew Bai +1
Data attribution methods trace model behavior back to its training dataset, offering an effective approach to better understand ''black-box'' neural networks. While prior research…