1 citations · 1 across the 10 of their papers we have counts for
6 papers · 1 filter
Memory Retrieval and Consolidation in Large Language Models through Function Tokens
Shaohua Zhang, Yuan Lin, Hang Li
The remarkable success of large language models (LLMs) stems from their ability to consolidate vast amounts of knowledge into the memory during pre-training and to retrieve it from…
Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Check
Chentao Cao, Xiaojun Xu, Bo Han +1
As large language models (LLMs) continue to advance in capabilities, ensuring their safety against jailbreak attacks remains a critical challenge. In this paper, we introduce a nov…
How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMs
Andrew Estornell, Jean-Francois Ton, Muhammad Faaiz Taufiq +1
Large Language Models (LLMs) have achieved strong performance on a wide range of complex reasoning tasks, yet further gains are often possible by leveraging the complementary stren…
LLM-VPRF: Large Language Model Based Vector Pseudo Relevance Feedback
Hang Li, Shengyao Zhuang, Bevan Koopman +1
Vector Pseudo Relevance Feedback (VPRF) has shown promising results in improving BERT-based dense retrieval systems through iterative refinement of query representations. This pape…
Pseudo Relevance Feedback is Enough to Close the Gap Between Small and Large Dense Retrieval Models
Hang Li, Xiao Wang, Bevan Koopman +1
Scaling dense retrievers to larger large language model (LLM) backbones has been a dominant strategy for improving their retrieval effectiveness. However, this has substantial cost…
Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models
Zhenghao Lin, Zihao Tang, Xiao Liu +31
We introduce Sigma, an efficient large language model specialized for the system domain, empowered by a novel architecture including DiffQKV attention, and pre-trained on our metic…