371 citations · 3.9k across the 110 of their papers we have counts for
6 papers · 1 filter
Multimodal Foundation Models: From Specialists to General-Purpose Assistants
Chunyuan Li, Zhe Gan, Zhengyuan Yang +4
This paper presents a comprehensive survey of the taxonomy and evolution of multimodal foundation models that demonstrate vision and vision-language capabilities, focusing on the t…
Guiding Instruction-based Image Editing via Multimodal Large Language Models
Tsu-Jui Fu, Wenze Hu, Xianzhi Du +3
Instruction-based image editing improves the controllability and flexibility of image manipulation via natural commands without elaborate descriptions or regional masks. However, h…
Reciprocating Magnetic Fields in the Pulsar Wind Observed from the Black Widow Pulsar J1720-0534
Chen-Chen Miao, Victoria Blackmon, Wei-Wei Zhu +41
We report the radio observations of the eclipsing black widow pulsar J1720-0534, a 3.26 ms pulsar in orbit with a low mass companion of mass 0.029 to 0.034 M. We obtain t…
MOFI: Learning Image Representations from Noisy Entity Annotated Images
Wentao Wu, Aleksei Timofeev, Chen Chen +8
We present MOFI, Manifold OF Images, a new vision foundation model designed to learn image representations from noisy entity annotated images. MOFI differs from previous work in tw…
An Empirical Study of Multimodal Model Merging
Yi-Lin Sung, Linjie Li, Kevin Lin +3
Model merging (e.g., via interpolation or task arithmetic) fuses multiple models trained on different tasks to generate a multi-task solution. The technique has been proven success…
Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation
Jaemin Cho, Linjie Li, Zhengyuan Yang +3
Spatial control is a core capability in controllable image generation. Advancements in layout-guided image generation have shown promising results on in-distribution (ID) datasets…