74 citations · 112 across the 12 of their papers we have counts for
17 papers
Diversity Matters: Revisiting Test-Time Compute in Vision-Language Models
Yijie Tong, Yifan Hou, Shaobo Cui +2
Test-time compute (TTC) strategies have emerged as a lightweight approach to boost reasoning in large language models (LLMs). However, their application and benefits for vision-lan…
Unveiling the Visual Counting Bottleneck in Vision-Language Models
Xingzhou Pang, Yifan Hou, Junling Wang +1
While Large Vision-Language Models (VLMs) excel at interpolation, they suffer catastrophic failures in systematic generalization, most notably in visual counting. In this work, we…
Chimera: Diagnosing Shortcut Learning in Visual-Language Understanding
Ziheng Chi, Yifan Hou, Chenxi Pang +3
Diagrams convey symbolic information in a visual format rather than a linear stream of words, making them especially challenging for AI models to process. While recent evaluations…
Can Vision-Language Models Solve Visual Math Equations?
Monjoy Narayan Choudhury, Junling Wang, Yifan Hou +1
Despite strong performance in visual understanding and language-based reasoning, Vision-Language Models (VLMs) struggle with tasks requiring integrated perception and symbolic comp…
Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning
Yucheng Wang, Yifan Hou, Aydin Javadov +2
Multimodal large language models (MLLMs) promise enhanced reasoning by integrating diverse inputs such as text, vision, and audio. Yet cross-modal reasoning remains underexplored,…
Do Vision-Language Models Really Understand Visual Language?
Yifan Hou, Buse Giledereli, Yilei Tu +1
Visual language is a system of communication that conveys information through symbols, shapes, and spatial arrangements. Diagrams are a typical example of a visual language depicti…
What Do Language Models Learn in Context? The Structured Task Hypothesis
Jiaoda Li, Yifan Hou, Mrinmaya Sachan +1
Large language models (LLMs) exhibit an intriguing ability to learn a novel task from in-context examples presented in a demonstration, termed in-context learning (ICL). Understand…
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
Yifan Hou, Jiaoda Li, Yu Fei +5
Recent work has shown that language models (LMs) have strong multi-step (i.e., procedural) reasoning capabilities. However, it is unclear whether LMs perform these tasks by cheatin…
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui +5
The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the r…
Mitigating Label Biases for In-context Learning
Yu Fei, Yifan Hou, Zeming Chen +1
Various design settings for in-context learning (ICL), such as the choice and order of the in-context examples, can bias a model toward a particular prediction without being reflec…
Adapters for Enhanced Modeling of Multilingual Knowledge and Text
Yifan Hou, Wenxiang Jiao, Meizhen Liu +3
Large language models appear to learn facts from the large text corpora they are trained on. Such facts are encoded implicitly within their many parameters, making it difficult to…
A Representation Learning Framework for Property Graphs
Yifan Hou, Hongzhi Chen, Changji Li +2
Representation learning on graphs, also called graph embedding, has demonstrated its significant impact on a series of machine learning applications such as classification, predict…
Measuring and Improving the Use of Graph Information in Graph Neural Networks
Yifan Hou, Jian Zhang, James Cheng +4
Graph neural networks (GNNs) have been widely used for representation learning on graph data. However, there is limited understanding on how much performance GNNs actually gain fro…
What Has Been Enhanced in my Knowledge-Enhanced Language Model?
Yifan Hou, Guoji Fu, Mrinmaya Sachan
Pretrained language models (LMs) do not capture factual knowledge very well. This has led to the development of a number of knowledge integration (KI) methods which aim to incorpor…
Bird's Eye: Probing for Linguistic Graph Structures with a Simple Information-Theoretic Approach
Yifan Hou, Mrinmaya Sachan
NLP has a rich history of representing our prior understanding of language in the form of graphs. Recent work on analyzing contextualized text representations has focused on hand-d…
Understanding Graph Neural Networks from Graph Signal Denoising Perspectives
Guoji Fu, Yifan Hou, Jian Zhang +3
Graph neural networks (GNNs) have attracted much attention because of their excellent performance on tasks such as node classification. However, there is inadequate understanding o…
Context-Aware Online Learning for Course Recommendation of MOOC Big Data
Yifan Hou, Pan Zhou, Ting Wang +3
The Massive Open Online Course (MOOC) has expanded significantly in recent years. With the widespread of MOOC, the opportunity to study the fascinating courses for free has attract…