7 papers · 1 filter
OASIS: Online Sample Selection for Continual Visual Instruction Tuning
Minjae Lee, Minhyuk Seo, Tingyu Qu +2
In continual instruction tuning (CIT) scenarios, where new instruction tuning data continuously arrive in an online streaming manner, training delays from large-scale data signific…
Towards More Accurate Personalized Image Generation: Addressing Overfitting and Evaluation Bias
Mingxiao Li, Tingyu Qu, Tinne Tuytelaars +1
Personalized image generation via text prompts has great potential to improve daily life and professional work by facilitating the creation of customized visual content. The aim of…
TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models
Tingyu Qu, Mingxiao Li, Tinne Tuytelaars +1
Recent advances in multimodal Large Language Models (LLMs) have shown great success in understanding multi-modal contents. For video understanding tasks, training-based video LLMs…
Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks
Tingyu Qu, Tinne Tuytelaars, Marie-Francine Moens
Mainstream parameter-efficient fine-tuning (PEFT) methods, such as LoRA or Adapter, project a model's hidden states to a lower dimension, allowing pre-trained models to adapt to ne…
Visually-Aware Context Modeling for News Image Captioning
Tingyu Qu, Tinne Tuytelaars, Marie-Francine Moens
News Image Captioning aims to create captions from news articles and images, emphasizing the connection between textual context and visual elements. Recognizing the significance of…
Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps
Mingxiao Li, Tingyu Qu, Ruicong Yao +2
Diffusion Probabilistic Models (DPM) have shown remarkable efficacy in the synthesis of high-quality images. However, their inference process characteristically requires numerous,…