4 papers
SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Models
Anushka Sivakumar, Andrew Zhang, Zaber Hakim +1
This work introduces SteerVLM, a lightweight steering module designed to guide Vision-Language Models (VLMs) towards outputs that better adhere to desired instructions. Our approac…
Flexible-length Text Infilling for Discrete Diffusion Models
Andrew Zhang, Anushka Sivakumar, Chiawei Tang +1
Discrete diffusion models are a new class of text generators that offer advantages such as bidirectional context use, parallelizable generation, and flexible prompting compared to…
Zero-Shot Fine-Grained Image Classification Using Large Vision-Language Models
Md. Atabuzzaman, Andrew Zhang, Chris Thomas
Large Vision-Language Models (LVLMs) have demonstrated impressive performance on vision-language reasoning tasks. However, their potential for zero-shot fine-grained image classifi…
Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval
Hani Alomari, Anushka Sivakumar, Andrew Zhang +1
Cross-modal image-text retrieval is challenging because of the diverse possible associations between content from different modalities. Traditional methods learn a single-vector em…