22 citations · 23 across the 13 of their papers we have counts for
17 papers
IsoVRBench: Video Reward Modeling Without Human Feedback
Shuyue Stella Li, Jacqueline He, Jiahui Chen +5
Video reward models are trained and evaluated on human preferences between videos that often differ simultaneously on properties such as prompt alignment, visual fidelity, motion,…
ExploreNet: Learning Where to Explore in Diffusion GRPO
Shuyue Stella Li, Xiaochuang Han, Yulia Tsvetkov +1
Group-relative RL methods such as Flow-GRPO post-train image generators by exploring with isotropic Gaussian noise added at every denoising step. This noise decides which rollouts…
Verifiable Visual Rewards Transfer from Synthetic Scenes to Natural Prompts
Shuyue Stella Li, Xiaochuang Han, Yulia Tsvetkov +1
Precise instruction following in image generation, such as satisfying object counts and spatial relations, remains an open challenge at least in part because it is learned using un…
WaiT for the Signal: Simple Frequency-Aware Flow-Matching
Krunoslav Lehman Pavasovic, Théophane Vallaeys, Stéphane Mallat +4
As image generation models scale to ever higher resolutions, global coherence, local detail, and texture fidelity become critical axes for generation quality. However, standard flo…
Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation
Zhiheng Liu, Weiming Ren, Xiaoke Huang +12
Unified multimodal models typically rely on pretrained vision encoders and use separate visual representations for understanding and generation, creating misalignment between the t…
Beyond Language Modeling: An Exploration of Multimodal Pretraining
Shengbang Tong, David Fan, John Nguyen +18
The visual world offers a critical axis for advancing foundation models beyond language. Despite growing interest in this direction, the design space for native multimodal models r…
Anchored Decoding: Provably Reducing Copyright Risk for Any Language Model
Jacqueline He, Jonathan Hayase, Wen-tau Yih +3
Language models (LMs) tend to memorize portions of their training data and emit verbatim spans. When the underlying sources are sensitive or copyright-protected, such reproduction…
TV2TV: A Unified Framework for Interleaved Language and Video Generation
Xiaochuang Han, Youssef Emad, Melissa Hall +15
Video generation models are rapidly advancing, but can still struggle with complex video outputs that require significant semantic branching or repeated high-level reasoning about…
DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research
Rulin Shao, Akari Asai, Shannon Zejiang Shen +18
Deep research agents perform multi-step research to produce long-form, well-attributed answers. However, most open deep research agents are trained on easily verifiable short-form…
Continual Learning via Sparse Memory Finetuning
Jessy Lin, Luke Zettlemoyer, Gargi Ghosh +4
Modern language models are powerful, but typically static after deployment. A major obstacle to building models that continually learn over time is catastrophic forgetting, where u…
OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows
John Nguyen, Marton Havasi, Tariq Berrada +2
We present OneFlow, the first non-autoregressive multimodal model that enables variable-length and concurrent mixed-modal generation. Unlike autoregressive models that enforce rigi…
Meta CLIP 2: A Worldwide Scaling Recipe
Yung-Sung Chuang, Yang Li, Dong Wang +13
Contrastive Language-Image Pretraining (CLIP) is a popular foundation model, supporting from zero-shot classification, retrieval to encoders for multimodal large language models (M…
Memory Layers at Scale
Vincent-Pierre Berges, Barlas Oğuz, Daniel Haziza +3
Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, sparsely activated memory layers complement comp…
Improving Factuality with Explicit Working Memory
Mingda Chen, Yang Li, Karthik Padthe +5
Large language models can generate factually inaccurate content, a problem known as hallucination. Recent works have built upon retrieved-augmented generation to improve factuality…
Altogether: Image Captioning via Re-aligning Alt-text
Hu Xu, Po-Yao Huang, Xiaoqing Ellen Tan +10
This paper focuses on creating synthetic data to improve the quality of image captions. Existing works typically have two shortcomings. First, they caption images from scratch, ign…
MoDE: CLIP Data Experts via Clustering
Jiawei Ma, Po-Yao Huang, Saining Xie +5
The success of contrastive language-image pretraining (CLIP) relies on the supervision from the pairing between images and captions, which tends to be noisy in web-crawled data. We…
Demystifying CLIP Data
Hu Xu, Saining Xie, Xiaoqing Ellen Tan +7
Contrastive Language-Image Pre-training (CLIP) is an approach that has advanced research and applications in computer vision, fueling modern recognition systems and generative mode…