Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024
Privacy-preserved LLM Cascade via CoT-enhanced Policy Learning
Kai Zhang, Congchao Wang, Liqian Peng +2
Large Language Models (LLMs) have gained significant attention in on-device applications due to their remarkable performance across real-world tasks. However, on-device LLMs often…
cs.CL2024
Cascade-Aware Training of Language Models
Congchao Wang, Sean Augenstein, Keith Rush +5
Reducing serving cost and latency is a fundamental concern for the deployment of language models (LMs) in business applications. To address this, cascades of LMs offer an effective…