3 citations · 7 across the 11 of their papers we have counts for
13 papers
Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning
Wen Ye, Ailin Deng, Zirui Wang +8
Visual mathematical reasoning can fail before reasoning begins: when a vision-language model (VLM) misreads a diagram, subsequent reasoning proceeds from false premises. This perce…
Source-Modality Monitoring in Vision-Language Models
Etha Tianze Hua, Tian Yun, Ellie Pavlick
We define and investigate source-modality monitoring -- the ability of multimodal models to track and communicate the input source from which pieces of information originate. We co…
Video Finetuning Improves Reasoning Between Frames
Ruiqi Yang, Tian Yun, Zihan Wang +1
Multimodal large language models (LLMs) have made rapid progress in visual understanding, yet their extension from images to videos often reduces to a naive concatenation of frame…
What is an "Abstract Reasoner"? Revisiting Experiments and Arguments about Large Language Models
Tian Yun, Chen Sun, Ellie Pavlick
Recent work has argued that large language models (LLMs) are not "abstract reasoners", citing their poor zero-shot performance on a variety of challenging tasks as evidence. We rev…
How Do Vision-Language Models Process Conflicting Information Across Modalities?
Tianze Hua, Tian Yun, Ellie Pavlick
AI models are increasingly required to be multimodal, integrating disparate input streams into a coherent state representation on which subsequent behaviors and actions can be base…
$100K or 100 Days: Trade-offs when Pre-Training with Academic Resources
Apoorv Khandelwal, Tian Yun, Nihal V. Nayak +4
Pre-training is notoriously compute-intensive and academic researchers are notoriously under-resourced. It is, therefore, commonly assumed that academics can't pre-train models. In…
mOthello: When Do Cross-Lingual Representation Alignment and Cross-Lingual Transfer Emerge in Multilingual Models?
Tianze Hua, Tian Yun, Ellie Pavlick
Many pretrained multilingual models exhibit cross-lingual transfer ability, which is often attributed to a learned language-neutral representation during pretraining. However, it r…
Pre-trained Vision-Language Models Learn Discoverable Visual Concepts
Yuan Zang, Tian Yun, Hao Tan +2
Do vision-language models (VLMs) pre-trained to caption an image of a "durian" learn visual concepts such as "brown" (color) and "spiky" (texture) at the same time? We aim to answe…
Emergence of Abstract State Representations in Embodied Sequence Modeling
Tian Yun, Zilai Zeng, Kunal Handa +4
Decision making via sequence modeling aims to mimic the success of language models, where actions taken by an embodied agent are modeled as tokens to predict. Despite their promisi…
Improved Inference of Human Intent by Combining Plan Recognition and Language Feedback
Ifrah Idrees, Tian Yun, Naveen Sharma +4
Conversational assistive robots can aid people, especially those with cognitive impairments, to accomplish various tasks such as cooking meals, performing exercises, or operating m…
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?
Tian Yun, Usha Bhalla, Ellie Pavlick +1
Vision-language (VL) pretrained models have achieved impressive performance on multimodal reasoning and zero-shot recognition tasks. Many of these VL models are pretrained on unlab…
Does Vision-and-Language Pretraining Improve Lexical Grounding?
Tian Yun, Chen Sun, Ellie Pavlick
Linguistic representations derived from text alone have been criticized for their lack of grounding, i.e., connecting words to their meanings in the physical world. Vision-and-Lang…