5 papers
Latent Thoughts Tuning: Bridging Context and Reasoning with Fused Information in Latent Tokens
Weihao Liu, Dehai Min, Lu Cheng
While explicit Chain-of-Thought (CoT) equips Large Language Models (LLMs) with strong reasoning capabilities, it constrains the model's thoughts to a discrete vocabulary space. Rec…
Verifiable Rewards Beyond Math and Code: Lightweight Corpus-Grounded Process Supervision for Factual Question Answering
Shicheng Fan, Haochang Hao, Dehai Min +3
Applying reinforcement learning to improve factual accuracy in knowledge-intensive question answering faces a reward design dilemma. Response-level rewards provide only coarse supe…
SPA: Towards A Computational Friendly Cloud-Base and On-Devices Collaboration Seq2seq Personalized Generation with Casual Inference
Yanming Liu, Xinyue Peng, Ningjing Sang +10
Large language models(LLMs) have shown its outperforming ability on various tasks and question answering. However, LLMs require substantial memory storage on low-resource devices.…
Selected Languages are All You Need for Cross-lingual Truthfulness Transfer
Weihao Liu, Ning Wu, Wenbiao Ding +3
Truthfulness stands out as an essential challenge for Large Language Models (LLMs). Although many works have developed various ways for truthfulness enhancement, they seldom focus…
MuDAF: Long-Context Multi-Document Attention Focusing through Contrastive Learning on Attention Heads
Weihao Liu, Ning Wu, Shiping Yang +4
Large Language Models (LLMs) frequently show distracted attention due to irrelevant information in the input, which severely impairs their long-context capabilities. Inspired by re…