1.5k citations · 2.6k across the 50 of their papers we have counts for
53 papers
What Matters for Latent Reasoning with Flow Matching
Yassine Ouali, Adrian Bulat, Georgios Tzimiropoulos
Latent reasoning lets a large language model (LLM) think in a continuous space and verbalize only the answer. We argue that an effective latent thought must meet five requirements:…
UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
Ioannis Maniadis Metaxas, Adrian Bulat, Alberto Baldrati +4
Large Vision-Language Models (LVLMs) remain bottlenecked by massive computational footprints, precluding their deployment on resource-constrained edge devices. While efforts to com…
Hierarchical Image Tokenization for Multi-Scale Image Super Resolution
Isma Hadji, Enrique Sanchez, Adrian Bulat +2
We introduce a multi-scale Image Super Resolution (ISR) method building on recent advances in Visual Auto-Regressive (VAR) modeling. VAR models break image tokenization into additi…
Restore, Assess, Repeat: A Unified Framework for Iterative Image Restoration
I-Hsiang Chen, Isma Hadji, Enrique Sanchez +5
Image restoration aims to recover high quality images from inputs degraded by various factors, such as adverse weather, blur, or low light. While recent studies have shown remarkab…
VISion On Request: Enhanced VLLM efficiency with sparse, dynamically selected, vision-language interactions
Adrian Bulat, Alberto Baldrati, Ioannis Maniadis Metaxas +2
Existing approaches for improving the efficiency of Large Vision-Language Models (LVLMs) are largely based on the concept of visual token reduction. This approach, however, creates…
More Images, More Problems? A Controlled Analysis of VLM Failure Modes
Anurag Das, Adrian Bulat, Alberto Baldrati +4
Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities, yet their proficiency in understanding and reasoning over multiple images remains largely unexplored…
Vision-Free Retrieval: Rethinking Multimodal Search with Textual Scene Descriptions
Ioanna Ntinou, Alexandros Xenos, Yassine Ouali +2
Contrastively-trained Vision-Language Models (VLMs), such as CLIP, have become the standard approach for learning discriminative vision-language representations. However, these mod…
Multi-scale Image Super Resolution with a Single Auto-Regressive Model
Enrique Sanchez, Isma Hadji, Adrian Bulat +3
In this paper we tackle Image Super Resolution (ISR), using recent advances in Visual Auto-Regressive (VAR) modeling. VAR iteratively estimates the residual in latent space between…
Fwd2Bot: LVLM Visual Token Compression with Double Forward Bottleneck
Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos
In this work, we aim to compress the vision tokens of a Large Vision Language Model (LVLM) into a representation that is simultaneously suitable for (a) generative and (b) discrimi…
VladVA: Discriminative Fine-tuning of LVLMs
Yassine Ouali, Adrian Bulat, Alexandros Xenos +4
Contrastively-trained Vision-Language Models (VLMs) like CLIP have become the de facto approach for discriminative vision-language representation learning. However, these models ha…
FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion
Haosen Yang, Adrian Bulat, Isma Hadji +4
Diffusion models are proficient at generating high-quality images. They are however effective only when operating at the resolution used during training. Inference at a scaled reso…
CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs
Yassine Ouali, Adrian Bulat, Brais Martinez +1
Despite recent successes, LVLMs or Large Vision Language Models are prone to hallucinating details like objects and their properties or relations, limiting their real-world deploym…
FFF: Fixing Flawed Foundations in contrastive pre-training results in very strong Vision-Language models
Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos
Despite noise and caption quality having been acknowledged as important factors impacting vision-language contrastive pre-training, in this paper, we show that the full potential o…
You Only Need One Step: Fast Super-Resolution with Stable Diffusion via Scale Distillation
Mehdi Noroozi, Isma Hadji, Brais Martinez +2
In this paper, we introduce YONOS-SR, a novel stable diffusion-based approach for image super-resolution that yields state-of-the-art results using only a single DDIM step. We prop…
Aligned Unsupervised Pretraining of Object Detectors with Self-training
Ioannis Maniadis Metaxas, Adrian Bulat, Ioannis Patras +2
The unsupervised pretraining of object detectors has recently become a key component of object detector training, as it leads to improved performance and faster convergence during…
Black Box Few-Shot Adaptation for Vision-Language models
Yassine Ouali, Adrian Bulat, Brais Martinez +1
Vision-Language (V-L) models trained with contrastive learning to align the visual and language modalities have been shown to be strong few-shot learners. Soft prompt learning is t…
FS-DETR: Few-Shot DEtection TRansformer with prompting and without re-training
Adrian Bulat, Ricardo Guerrero, Brais Martinez +1
This paper is on Few-Shot Object Detection (FSOD), where given a few templates (examples) depicting a novel class (not seen during training), the goal is to detect all of its occur…
Bayesian Prompt Learning for Image-Language Model Generalization
Mohammad Mahdi Derakhshani, Enrique Sanchez, Adrian Bulat +4
Foundational image-language models have generated considerable interest due to their efficient adaptation to downstream tasks by prompt learning. Prompt learning treats part of the…
LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Models
Adrian Bulat, Georgios Tzimiropoulos
Soft prompt learning has recently emerged as one of the methods of choice for adapting V&L models to a downstream task using a few training examples. However, current methods signi…
REST: REtrieve & Self-Train for generative action recognition
Adrian Bulat, Enrique Sanchez, Brais Martinez +1
This work is on training a generative action/video recognition model whose output is a free-form action-specific caption describing the video (rather than an action class label). A…
Efficient Attention-free Video Shift Transformers
Adrian Bulat, Brais Martinez, Georgios Tzimiropoulos
This paper tackles the problem of efficient video recognition. In this area, video transformers have recently dominated the efficiency (top-1 accuracy vs FLOPs) spectrum. At the sa…
iBoot: Image-bootstrapped Self-Supervised Video Representation Learning
Fatemeh Saleh, Fuwen Tan, Adrian Bulat +2
Learning visual representations through self-supervision is an extremely challenging task as the network needs to sieve relevant patterns from spurious distractors without the acti…
Knowledge Distillation Meets Open-Set Semi-Supervised Learning
Jing Yang, Xiatian Zhu, Adrian Bulat +2
Existing knowledge distillation methods mostly focus on distillation of teacher's prediction and intermediate activation. However, the structured representation, which arguably is…
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers
Junting Pan, Adrian Bulat, Fuwen Tan +5
Self-attention based models such as vision transformers (ViTs) have emerged as a very competitive architecture alternative to convolutional neural networks (CNNs) in computer visio…