3 citations · 9 across the 12 of their papers we have counts for
12 papers
Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning
Shaotian Yan, Kaiyuan Liu, Chen Shen +6
In this report, we introduce DASD-4B-Thinking, a lightweight yet highly capable, fully open-source reasoning model. It achieves SOTA performance among open-source models of compara…
Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
Kaiyuan Liu, Shaotian Yan, Rui Miao +4
Reasoning distillation has attracted increasing attention. It typically leverages a large teacher model to generate reasoning paths, which are then used to fine-tune a student mode…
SalaMAnder: Shapley-based Mathematical Expression Attribution and Metric for Chain-of-Thought Reasoning
Yue Xin, Chen Shen, Shaotian Yan +5
Chain-of-Thought (CoT) prompting enhances the math reasoning capability of large language models (LLMs) to a large margin. However, the mechanism underlying such improvements remai…
Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning
Chenxi Huang, Shaotian Yan, Liang Xie +6
Representation Fine-tuning (ReFT), a recently proposed Parameter-Efficient Fine-Tuning (PEFT) method, has attracted widespread attention for significantly improving parameter effic…
Enhancing Large Language Models with Reward-guided Tree Search for Knowledge Graph Question and Answering
Xiao Long, Liansheng Zhuang, Chen Shen +3
Recently, large language models (LLMs) have demonstrated impressive performance in Knowledge Graph Question Answering (KGQA) tasks, which aim to find answers based on knowledge gra…
Improving Complex Reasoning with Dynamic Prompt Corruption: A soft prompt Optimization Approach
Sinan Fan, Liang Xie, Chen Shen +7
Prompt-tuning (PT) for large language models (LLMs) can facilitate the performance on various conventional NLP tasks with significantly fewer trainable parameters. However, our inv…
Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models
Shaotian Yan, Chen Shen, Wenxiao Wang +3
Few-shot Chain-of-Thought (CoT) significantly enhances the reasoning capabilities of large language models (LLMs), functioning as a whole to guide these models in generating reason…
Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs
Xiaofeng Zhang, Yihao Quan, Chaochen Gu +6
The hallucination problem in multimodal large language models (MLLMs) remains a common issue. Although image tokens occupy a majority of the input sequence of MLLMs, there is limit…
Instance-adaptive Zero-shot Chain-of-Thought Prompting
Xiaosong Yuan, Chen Shen, Shaotian Yan +6
Zero-shot Chain-of-Thought (CoT) prompting emerges as a simple and effective strategy for enhancing the performance of large language models (LLMs) in real-world reasoning tasks. N…
SciPIP: An LLM-based Scientific Paper Idea Proposer
Wenxiao Wang, Lihui Gu, Liye Zhang +7
The rapid advancement of large language models (LLMs) has opened new possibilities for automating the proposal of innovative scientific ideas. This process involves two key phases:…
A3S: A General Active Clustering Method with Pairwise Constraints
Xun Deng, Junlong Liu, Han Zhong +5
Active clustering aims to boost the clustering performance by integrating human-annotated pairwise constraints through strategic querying. Conventional approaches with semi-supervi…
From Redundancy to Relevance: Information Flow in LVLMs Across Reasoning Tasks
Xiaofeng Zhang, Yihao Quan, Chen Shen +7
Large Vision Language Models (LVLMs) achieve great performance on visual-language reasoning tasks, however, the black-box nature of LVLMs hinders in-depth research on the reasoning…