17 citations · 23 across the 18 of their papers we have counts for
6 papers · 1 filter
Training LLMs to be Better Text Embedders through Bidirectional Reconstruction
Chang Su, Dengliang Shi, Siyuan Huang +5
Large language models (LLMs) have increasingly been explored as powerful text embedders. Existing LLM-based text embedding approaches often leverage the embedding of the final toke…
AWM: Accurate Weight-Matrix Fingerprint for Large Language Models
Boyi Zeng, Lin Chen, Ziwei He +2
Protecting the intellectual property of large language models (LLMs) is crucial, given the substantial resources required for their training. Consequently, there is an urgent need…
PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space
Boyi Zeng, He Li, Shixiang Song +5
The remarkable success of Chain-of-Thought (CoT), which enhances performance by scaling generation steps at test-time, inspires us to ask: can we leverage a similar scaling of comp…
PonderLM: Pretraining Language Models to Ponder in Continuous Space
Boyi Zeng, Shixiang Song, Siyuan Huang +6
Humans ponder before articulating complex sentence elements, enabling deeper cognitive processing through focused effort. In this work, we introduce this pondering process into lan…
FreqKV: Key-Value Compression in Frequency Domain for Context Window Extension
Jushi Kai, Yixuan Wang, Boyi Zeng +4
Existing key-value (KV) cache compression methods for large language models (LLMs) often rely on token eviction, which risks losing critical local information in both long prefilli…
Gumbel Reranking: Differentiable End-to-End Reranker Optimization
Siyuan Huang, Zhiyuan Ma, Jintao Du +5
RAG systems rely on rerankers to identify relevant documents. However, fine-tuning these models remains challenging due to the scarcity of annotated query-document pairs. Existing…