17 citations · 23 across the 18 of their papers we have counts for
18 papers
DoPR: Reusable Compressed Document Prefixes for Efficient LLM Reranking
Beiya Dai, Yifan Wei, Guang Yang +3
Large language models (LLMs) are effective rerankers, but pointwise reranking repeatedly processes the same document across different queries, causing substantial redundant documen…
Pretraining with Token-Level Adaptive Latent Chain-of-Thought
Boyi Zeng, Yiqin Hao, He Li +8
Scaling large language models by increasing parameters and training data is increasingly constrained by limited high-quality corpora and rising communication costs. This work explo…
Flow of Spans: Generalizing Language Models to Dynamic Span-Vocabulary via GFlowNets
Bo Xue, Yunchong Song, Fanghao Shao +5
Standard autoregressive language models generate text token-by-token from a fixed vocabulary, inducing a tree-structured state space when viewing token sampling as an action, which…
Training LLMs to be Better Text Embedders through Bidirectional Reconstruction
Chang Su, Dengliang Shi, Siyuan Huang +5
Large language models (LLMs) have increasingly been explored as powerful text embedders. Existing LLM-based text embedding approaches often leverage the embedding of the final toke…
AWM: Accurate Weight-Matrix Fingerprint for Large Language Models
Boyi Zeng, Lin Chen, Ziwei He +2
Protecting the intellectual property of large language models (LLMs) is crucial, given the substantial resources required for their training. Consequently, there is an urgent need…
PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space
Boyi Zeng, He Li, Shixiang Song +5
The remarkable success of Chain-of-Thought (CoT), which enhances performance by scaling generation steps at test-time, inspires us to ask: can we leverage a similar scaling of comp…
PonderLM: Pretraining Language Models to Ponder in Continuous Space
Boyi Zeng, Shixiang Song, Siyuan Huang +6
Humans ponder before articulating complex sentence elements, enabling deeper cognitive processing through focused effort. In this work, we introduce this pondering process into lan…
FreqKV: Key-Value Compression in Frequency Domain for Context Window Extension
Jushi Kai, Yixuan Wang, Boyi Zeng +4
Existing key-value (KV) cache compression methods for large language models (LLMs) often rely on token eviction, which risks losing critical local information in both long prefilli…
Gumbel Reranking: Differentiable End-to-End Reranker Optimization
Siyuan Huang, Zhiyuan Ma, Jintao Du +5
RAG systems rely on rerankers to identify relevant documents. However, fine-tuning these models remains challenging due to the scarcity of annotated query-document pairs. Existing…
Enhancing SPARQL Generation by Triplet-order-sensitive Pre-training
Chang Su, Jiexing Qi, He Yan +2
Semantic parsing that translates natural language queries to SPARQL is of great importance for Knowledge Graph Question Answering (KGQA) systems. Although pre-trained language mode…
Mirror-Consistency: Harnessing Inconsistency in Majority Voting
Siyuan Huang, Zhiyuan Ma, Jintao Du +3
Self-Consistency, a widely-used decoding strategy, significantly boosts the reasoning capabilities of Large Language Models (LLMs). However, it depends on the plurality voting rule…
Cluster-wise Graph Transformer with Dual-granularity Kernelized Attention
Siyuan Huang, Yunchong Song, Jiayue Zhou +1
In the realm of graph learning, there is a category of methods that conceptualize graphs as hierarchical structures, utilizing node clustering to capture broader structural informa…
Graph Parsing Networks
Yunchong Song, Siyuan Huang, Xinbing Wang +2
Graph pooling compresses graph information into a compact representation. State-of-the-art graph pooling methods follow a hierarchical approach, which reduces the graph size step-b…
SH2: Self-Highlighted Hesitation Helps You Decode More Truthfully
Jushi Kai, Tianhang Zhang, Hai Hu +1
Large language models (LLMs) demonstrate great performance in text generation. However, LLMs are still suffering from hallucinations. In this work, we propose an inference-time met…
GeoGalactica: A Scientific Large Language Model in Geoscience
Zhouhan Lin, Cheng Deng, Le Zhou +18
Large language models (LLMs) have achieved huge success for their general knowledge and ability to solve a wide spectrum of tasks in natural language processing (NLP). Due to their…
Towards Controlled Table-to-Text Generation with Scientific Reasoning
Zhixin Guo, Jianping Zhou, Jiexing Qi +6
The sheer volume of scientific experimental results and complex technical statements, often presented in tabular formats, presents a formidable barrier to individuals acquiring pre…
HuRef: HUman-REadable Fingerprint for Large Language Models
Boyi Zeng, Lizheng Wang, Yuncong Hu +5
Protecting the copyright of large language models (LLMs) has become crucial due to their resource-intensive training and accompanying carefully designed licenses. However, identify…
Tailoring Self-Attention for Graph via Rooted Subtrees
Siyuan Huang, Yunchong Song, Jiayue Zhou +1
Attention mechanisms have made significant strides in graph learning, yet they still exhibit notable limitations: local attention faces challenges in capturing long-range informati…