Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

Abstract

We present the Qwen2-VL Series, an advanced upgrade of the previous Qwen-VL models that redefines the conventional predetermined-resolution approach in visual processing. Qwen2-VL introduces the Naive Dynamic Resolution mechanism, which enables the model to dynamically process images of varying resolutions into different numbers of visual tokens. This approach allows the model to generate more efficient and accurate visual representations, closely aligning with human perceptual processes. The model also integrates Multimodal Rotary Position Embedding (M-RoPE), facilitating the effective fusion of positional information across text, images, and videos. We employ a unified paradigm for processing both images and videos, enhancing the model's visual perception capabilities. To explore the potential of large multimodal models, Qwen2-VL investigates the scaling laws for large vision-language models (LVLMs). By scaling both the model size-with versions at 2B, 8B, and 72B parameters-and the amount of training data, the Qwen2-VL Series achieves highly competitive performance. Notably, the Qwen2-VL-72B model achieves results comparable to leading models such as GPT-4o and Claude3.5-Sonnet across various multimodal benchmarks, outperforming other generalist models. Code is available at https://github.com/QwenLM/Qwen2-VL .

Code is available at https://github.com/QwenLM/Qwen2-VL. arXiv admin note: text overlap with arXiv:2408.15262 by other authors

Cited by in corpus (11)

• Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation
• Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision
• Reasoning Beyond Limits: Advances and Open Problems for LLMs
• Human-like object concept representations emerge naturally in multimodal large language models
• Object Detection with Multimodal Large Vision-Language Models: An In-depth Review
• EmoVerse: Exploring Multimodal Large Language Models for Sentiment and Emotion Understanding
• EH-Benchmark Ophthalmic Hallucination Benchmark and Agent-Driven Top-Down Traceable Reasoning Workflow
• MULTI: Multimodal Understanding Leaderboard with Text and Images
• JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering
• EEmo-Bench: A Benchmark for Multi-modal Large Language Models on Image Evoked Emotion Assessment
• BoundingDocs: a Unified Dataset for Document Question Answering with Spatial Annotations