6 papers
Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models
Boyi Deng, Xu Wang, Yaoning Wang +15
Large language models have achieved remarkable capabilities across diverse tasks, yet their internal decision-making processes remain largely opaque, limiting our ability to inspec…
SASFT: Sparse Autoencoder-guided Supervised Finetuning to Mitigate Unexpected Code-Switching in LLMs
Boyi Deng, Yu Wan, Baosong Yang +3
Large Language Models (LLMs) have impressive multilingual capabilities, but they suffer from unexpected code-switching, also known as language mixing, which involves switching to u…
Controllable LLM Reasoning via Sparse Autoencoder-Based Steering
Yi Fang, Wenjie Wang, Mingfeng Xue +4
Large Reasoning Models (LRMs) exhibit human-like cognitive reasoning strategies (\eg backtracking, cross-verification) during the reasoning process, which improves their performanc…
Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoders
Boyi Deng, Yu Wan, Yidan Zhang +2
The mechanisms behind multilingual capabilities in Large Language Models (LLMs) have been examined using neuron-based or internal-activation-based methods. However, these methods o…
P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs
Yidan Zhang, Yu Wan, Boyi Deng +6
Recent advancements in large language models (LLMs) showcase varied multilingual capabilities across tasks like translation, code generation, and reasoning. Previous assessments of…
CrAM: Credibility-Aware Attention Modification in LLMs for Combating Misinformation in RAG
Boyi Deng, Wenjie Wang, Fengbin Zhu +2
Retrieval-Augmented Generation (RAG) can alleviate hallucinations of Large Language Models (LLMs) by referencing external documents. However, the misinformation in external documen…