13 papers
Canonical Color as a Lens into Concept Decodability in Vision Encoders and VLMs
Xiaofu Chen, Stella Frank, Yova Kementchedjhieva
Visual encoders construct a representation of the image input for Vision-Language models. How much conceptual, as opposed to immediately visible, information does this representati…
Failing to See or Failing to Know? Attributing Errors in Vision-Language Models
Khang Nhat Hoang Vo, Artem Vazhentsev, Artem Shelmanov +2
Vision-language models (VLMs) can recognize entities in clear images yet still fail when answering questions that require factual knowledge beyond what is directly observable. Prio…
CRAFT: Cost-aware Refinement And Front-aware Tuning of Prompts
Shanu Kumar, Shubhanshu Khandelwal, Akhila Yesantarao Venkata +3
Prompts tuned for accuracy often grow long, raising inference cost on every model call. The best accuracy-cost trade-off depends on the task and the budget, so prompt optimization…
Pixel-TTS: Image based Text Rendering for Robust Text-to-Speech
Adarsh Arigala, Arjun Gangwar, S Umesh +1
Recent advances in pixel-based text modeling show that representing text as images enables models to exploit visual cues for language understanding. Grounding text in its visual fo…
LinguDistill: Recovering Linguistic Ability in Vision-Language Models via Selective Cross-Modal Distillation
Patrick Amadeus Irawan, Erland Hilman Fuadi, Shanu Kumar +2
Adapting pretrained language models (LMs) into vision-language models (VLMs) can degrade their native linguistic capability due to representation shift and cross-modal interference…
VLMs Need Words: Vision Language Models Ignore Visual Detail In Favor of Semantic Anchors
Haz Sameen Shahgir, Xiaofu Chen, Yu Fu +4
Vision-language models (VLMs) have achieved impressive performance across a wide range of multimodal tasks. However, they often fail on tasks that require fine-grained visual perce…