1 citations · 1 across the 5 of their papers we have counts for
6 papers
Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment
Yihuai Hong, Shauli Ravfogel, Chen Zhao +1
Chain-of-thought (CoT) traces often serve as a proxy for how Large Language Models (LLMs) arrive at their answers. However, growing evidence shows that models' CoT often fails to r…
Training Continuous Chain of Thought Models: A Tale of Two Regimes
Varun Yerram, He He, Eunsol Choi
Continuous Chain-of-Thought methods replace verbose reasoning traces with a short sequence of dense latent representations. Earlier continuous CoT methods indirectly supervise the…
Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graph
Zhenheng Tang, Xiang Liu, Qian Wang +3
As Large Language Models (LLMs) become more powerful and autonomous, they increasingly face conflicts and dilemmas in many scenarios. We first summarize and taxonomize these divers…
Identifying Crucial Attention Heads for Multilingual Language Models: Retrieval and Retrieval-Transition Heads
Shaswat Patel, Vishvesh Trivedi, Yue Han +2
Retrieval heads, a subset of attention heads in Transformers, were studied in English, showing its crucial role in retrieving information from the context. We expand the study of r…
Beyond Single Embeddings: Capturing Diverse Targets with Multi-Query Retrieval
Hung-Ting Chen, Xiang Liu, Shauli Ravfogel +1
Most text retrievers generate \emph{one} query vector to retrieve relevant documents. Yet, the conditional distribution of relevant documents for the query may be multimodal, e.g.,…
DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference
Xiang Liu, Xuming Hu, Xiaowen Chu +1
Recent reasoning Large Language Models (LLMs) demonstrate remarkable problem-solving abilities but often generate long thinking traces whose utility is unclear. Our work aims to im…