11 papers
VIF-Bench: Evaluating Visual Instruction Following in Multi-Reference Image Generation
Yuta Oshima, Masakazu Yoshimura, Masahiro Suzuki +2
Recent multimodal image generation models can take multiple images and textual instructions as input, enabling reference-based generation guided not only by text but also by visual…
AutoRef: Harness Optimization for Agentic Multi-Reference Image Generation
Yuta Oshima, Ku Onoda, Yusuke Iwasawa +3
Recent image generation models can take multiple reference images as input and combine them into a new image. However, multi-reference image generation remains challenging: models…
Steering Recurrent Reasoners at Inference Time with Readout Feedback
Shunsuke Kamiya, Masanori Koyama, Seongcheol Jeong +5
Recurrent models, which repeatedly update latent states with shared computation blocks, have emerged as powerful architectures for solving complex reasoning tasks. Existing inferen…
What Keeps Vision-Language Models Looking at the Image?
Hiroto Osaka, Shohei Taniguchi, Gouki Minegishi +3
When do vision-language models need direct access to the image while generating an answer? We study image dependence during answer generation by examining how the visual informatio…
SUNTA: Hierarchical Video Prediction with Surprise-based Chunking
Tomoshi Iiyama, Masahiro Suzuki, Yutaka Matsuo
Hierarchical state-space models (HSSMs) offer a promising approach to long-horizon prediction by segmenting sequences into temporal chunks. However, their performance hinges on how…
WorldPack: Dynamic Frame Compression for Long-context Video World Modeling
Yuta Oshima, Yusuke Iwasawa, Masahiro Suzuki +2
Video world models have attracted significant attention for their ability to produce high-fidelity future visual observations conditioned on past observations and navigation action…
When Object-Centric World Models Meet Policy Learning: From Pixels to Policies, and Where It Breaks
Stefano Ferraro, Akihiro Nakano, Masahiro Suzuki +1
Object-centric world models (OCWM) aim to decompose visual scenes into object-level representations, providing structured abstractions that could improve compositional generalizati…
MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generation
Yuta Oshima, Daiki Miyake, Kohsei Matsutani +4
Recent text-to-image generation models have acquired the ability of multi-reference generation and editing; that is, to inherit the appearance of subjects from multiple reference i…
CLIP-like Model as a Foundational Density Ratio Estimator
Fumiya Uchiyama, Rintaro Yanagi, Shohei Taniguchi +5
Density ratio estimation is a core concept in statistical machine learning because it provides a unified mechanism for tasks such as importance weighting, divergence estimation, an…
ADOPT: Modified Adam Can Converge with Any with the Optimal Rate
Shohei Taniguchi, Keno Harada, Gouki Minegishi +7
Adam is one of the most popular optimization algorithms in deep learning. However, it is known that Adam does not converge in theory unless choosing a hyperparameter, i.e., ,…
Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biases
Cristian Meo, Akihiro Nakano, Mircea Lică +7
Unsupervised object-centric learning from videos is a promising approach towards learning compositional representations that can be applied to various downstream tasks, such as pre…