Showing cs.CLShow all
3 papers · 1 filter
cs.CL2025
Train It and Forget It: Merge Lists are Unnecessary for BPE Inference in Language Models
Tomohiro Sawada, Kartik Goyal
Standard Byte-Pair Encoding (BPE) tokenization compresses text by pairing a learned token vocabulary with a detailed merge list. Recent work has shown that this merge list exposes…
cs.CL2025
Cascaded Information Disclosure for Generalized Evaluation of Problem Solving Capabilities
Yunxiang Yan, Tomohiro Sawada, Kartik Goyal
While question-answering~(QA) benchmark performance is an automatic and scalable method to compare LLMs, it is an indirect method of evaluating their underlying problem-solving cap…
cs.CL2023
Towards a Unified Multimodal Reasoning Framework
Abhinav Arun, Dipendra Singh Mal, Mehul Soni +1
Recent advancements in deep learning have led to the development of powerful language models (LMs) that excel in various tasks. Despite these achievements, there is still room for…