5 citations · 5 across the 2 of their papers we have counts for
4 papers · 1 filter
Efficient Speculative Decoding for Llama at Scale: Challenges and Solutions
Bangsheng Tang, Carl Chengyan Fu, Fei Kou +35
Speculative decoding is a standard method for accelerating the inference speed of large language models. However, scaling it for production environments poses several engineering c…
When Every Token Counts: Optimal Segmentation for Low-Resource Language Models
Bharath Raj, Garvit Suri, Vikrant Dewangan +1
Traditional greedy tokenization methods have been a critical step in Natural Language Processing (NLP), influencing how text is converted into tokens and directly impacting model p…
Exploring Model Kinship for Merging Large Language Models
Yedi Hu, Yunzhi Yao, Ningyu Zhang +2
Model merging has emerged as a key technique for enhancing the capabilities and efficiency of Large Language Models (LLMs). The open-source community has driven model evolution by…
CMU LiveMedQA at TREC 2017 LiveQA: A Consumer Health Question Answering System
Yuan Yang, Jingcheng Yu, Ye Hu +2
In this paper, we present LiveMedQA, a question answering system that is optimized for consumer health question. On top of the general QA system pipeline, we introduce several new…