6 papers
Quantifying and Optimizing Simplicity via Polynomial Representations
Tianren Zhang, Xiangxin Li, Minghao Xiao +2
Deep networks often exhibit a preference for "simple" solutions, and such a simplicity bias is widely believed to play a key role in generalization. Yet a broadly applicable, quant…
DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoning
Tianrun Xu, Haoda Jing, Ye Li +6
Recent advances in multimodal language models (MLLMs) have made thinking with images a dominant paradigm for multimodal reasoning. However, existing methods still fail to ensure ev…
Reconciling In-Context and In-Weight Learning via Dual Representation Space Encoding
Guanyu Chen, Ruichen Wang, Tianren Zhang +1
In-context learning (ICL) is a valuable capability exhibited by Transformers pretrained on diverse sequence tasks. However, previous studies have observed that ICL often conflicts…
Exploring the Hidden Reasoning Process of Large Language Models by Misleading Them
Guanyu Chen, Peiyang Wang, Yizhou Jiang +5
Large language models (LLMs) have been able to perform various forms of reasoning tasks in a wide range of scenarios, but are they truly engaging in task abstraction and rule-based…
When Do Neural Networks Learn World Models?
Tianren Zhang, Guanyu Chen, Feng Chen
Humans develop world models that capture the underlying generation process of data. Whether neural networks can learn similar world models remains an open problem. In this work, we…
Feature contamination: Neural networks learn uncorrelated features and fail to generalize
Tianren Zhang, Chujie Zhao, Guanyu Chen +2
Learning representations that generalize under distribution shifts is critical for building robust machine learning models. However, despite significant efforts in recent years, al…