4 papers
Quantifying Positional Biases in Text Embedding Models
Reagan J. Lee, Samarth Goel, Kannan Ramchandran
Embedding models are crucial for tasks in Information Retrieval (IR) and semantic similarity measurement, yet their handling of longer texts and associated positional biases remain…
SAGE: A Realistic Benchmark for Semantic Understanding
Samarth Goel, Reagan J. Lee, Kannan Ramchandran
As large language models (LLMs) achieve strong performance on traditional benchmarks, there is an urgent need for more challenging evaluation frameworks that probe deeper aspects o…
VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data
Thomas Zeng, Shuibai Zhang, Shutong Wu +13
Process Reward Models (PRMs) have proven effective at enhancing mathematical reasoning for Large Language Models (LLMs) by leveraging increased inference-time computation. However,…
Looped Transformers for Length Generalization
Ying Fan, Yilun Du, Kannan Ramchandran +1
Recent work has shown that Transformers trained from scratch can successfully solve various arithmetic and algorithmic tasks, such as adding numbers and computing parity. While the…