4 papers
ABBEL: Learning Natural-Language Belief States for Memory-Efficient Interaction
Aly Lidayan, Jakob Bjorner, Satvik Golechha +2
As the time horizons of sequential decision-making tasks grow, keeping full interaction histories in model context becomes increasingly costly. Recent work reduces context lengths…
Semantic Differentiation for Tackling Challenges in Watermarking Low-Entropy Constrained Generation Outputs
Nghia T. Le, Alan Ritter, Kartik Goyal
We demonstrate that while the current approaches for language model watermarking are effective for open-ended generation, they are inadequate at watermarking LM outputs for constra…
Train It and Forget It: Merge Lists are Unnecessary for BPE Inference in Language Models
Tomohiro Sawada, Kartik Goyal
Standard Byte-Pair Encoding (BPE) tokenization compresses text by pairing a learned token vocabulary with a detailed merge list. Recent work has shown that this merge list exposes…
Cascaded Information Disclosure for Generalized Evaluation of Problem Solving Capabilities
Yunxiang Yan, Tomohiro Sawada, Kartik Goyal
While question-answering~(QA) benchmark performance is an automatic and scalable method to compare LLMs, it is an indirect method of evaluating their underlying problem-solving cap…