12 citations · 20 across the 23 of their papers we have counts for
11 papers · 1 filter
Steering Large Language Models to Evaluate and Amplify Creativity
Matthew Lyle Olson, Neale Ratzlaff, Musashi Hinck +2
Although capable of generating creative text, Large Language Models (LLMs) are poor judges of what constitutes "creativity". In this work, we show that we can leverage this knowled…
Training-Free Mitigation of Language Reasoning Degradation After Multimodal Instruction Tuning
Neale Ratzlaff, Man Luo, Xin Su +2
Multimodal models typically combine a powerful large language model (LLM) with a vision encoder and are then trained on multimodal data via instruction tuning. While this process a…
FiVL: A Framework for Improved Vision-Language Alignment through the Lens of Training, Evaluation and Explainability
Estelle Aflalo, Gabriela Ben Melech Stan, Tiep Le +5
Large Vision Language Models (LVLMs) have achieved significant progress in integrating visual and textual inputs for multimodal reasoning. However, a recurring challenge is ensurin…
A Causal World Model Underlying Next Token Prediction: Exploring GPT in a Controlled Environment
Raanan Y. Rohekar, Yaniv Gurwicz, Sungduk Yu +2
Are generative pre-trained transformer (GPT) models, trained only to predict the next token, implicitly learning a world model from which sequences are generated one token at a tim…
FastRM: An efficient and automatic explainability framework for multimodal generative models
Gabriela Ben-Melech Stan, Estelle Aflalo, Man Luo +5
Large Vision Language Models (LVLMs) have demonstrated remarkable reasoning capabilities over textual and visual inputs. However, these models remain prone to generating misinforma…
Debias your Large Multi-Modal Model at Test-Time via Non-Contrastive Visual Attribute Steering
Neale Ratzlaff, Matthew Lyle Olson, Musashi Hinck +4
Large Multi-Modal Models (LMMs) have demonstrated impressive capabilities as general-purpose chatbots able to engage in conversations about visual inputs. However, their responses…
Debiasing Large Vision-Language Models by Ablating Protected Attribute Representations
Neale Ratzlaff, Matthew Lyle Olson, Musashi Hinck +3
Large Vision Language Models (LVLMs) such as LLaVA have demonstrated impressive capabilities as general-purpose chatbots that can engage in conversations about a provided input ima…
Scaling Knowledge Graph Construction through Synthetic Data Generation and Distillation
Prafulla Kumar Choubey, Xin Su, Man Luo +9
Document-level knowledge graph (KG) construction faces a fundamental scaling challenge: existing methods either rely on expensive large language models (LLMs), making them economic…
Is Your Paper Being Reviewed by an LLM? Investigating AI Text Detectability in Peer Review
Sungduk Yu, Man Luo, Avinash Madasu +2
Peer review is a critical process for ensuring the integrity of published scientific research. Confidence in this process is predicated on the assumption that experts in the releva…
ClimDetect: A Benchmark Dataset for Climate Change Detection and Attribution
Sungduk Yu, Brian L. White, Anahita Bhiwandiwalla +6
Detecting and attributing temperature increases driven by climate change is crucial for understanding global warming and informing adaptation strategies. However, distinguishing hu…
Why do LLaVA Vision-Language Models Reply to Images in English?
Musashi Hinck, Carolin Holtermann, Matthew Lyle Olson +6
We uncover a surprising multilingual bias occurring in a popular class of multimodal vision-language models (VLMs). Including an image in the query to a LLaVA-style VLM significant…