15 citations · 41 across the 13 of their papers we have counts for
4 papers · 1 filter
HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
Howard Yen, Tianyu Gao, Minmin Hou +5
Many benchmarks exist for evaluating long-context language models (LCLMs), yet developers often rely on synthetic tasks such as needle-in-a-haystack (NIAH) or an arbitrary subset o…
How to Train Long-Context Language Models (Effectively)
Tianyu Gao, Alexander Wettig, Howard Yen +1
We study continued training and supervised fine-tuning (SFT) of a language model (LM) to make effective use of long-context information. We first establish a reliable evaluation pr…
BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
Hongjin Su, Howard Yen, Mengzhou Xia +12
Existing retrieval benchmarks primarily consist of information-seeking queries (e.g., aggregated questions from search engines) where keyword or semantic-based retrieval is usually…
Long-Context Language Modeling with Parallel Context Encoding
Howard Yen, Tianyu Gao, Danqi Chen
Extending large language models (LLMs) to process longer inputs is crucial for a wide range of applications. However, the substantial computational cost of transformers and limited…