11 citations · 22 across the 23 of their papers we have counts for
8 papers · 1 filter
Reusing Pre-Training Data at Test Time is a Compute Multiplier
Alex Fang, Thomas Voice, Ruoming Pang +2
Large language models learn from their vast pre-training corpora, gaining the ability to solve an ever increasing variety of tasks; yet although researchers work to improve these d…
Synthetic bootstrapped pretraining
Zitong Yang, Aonan Zhang, Hong Liu +4
We introduce Synthetic Bootstrapped Pretraining (SBP), a language model (LM) pretraining procedure that first learns a model of relations between documents from the pretraining dat…
Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge?
Arduin Findeis, Floris Weers, Guoli Yin +3
Pairwise preferences over model responses are widely collected to evaluate and provide feedback to large language models (LLMs). Given two alternative model responses to the same i…
RATTENTION: Towards the Minimal Sliding Window Size in Local-Global Attention Models
Bailin Wang, Chang Lan, Chong Wang +1
Local-global attention models have recently emerged as compelling alternatives to standard Transformers, promising improvements in both training and inference efficiency. However,…
Instruction-Following Pruning for Large Language Models
Bairu Hou, Qibin Chen, Jianyu Wang +6
With the rapid scaling of large language models (LLMs), structured pruning has become a widely used technique to learn efficient, smaller models from larger ones, delivering superi…
ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities
Jiarui Lu, Thomas Holleis, Yizhe Zhang +9
Recent large language models (LLMs) advancements sparked a growing research interest in tool assisted LLMs solving real-world challenges, which calls for comprehensive evaluation o…