5 papers
Type-Compliant Adaptation Cascades: Adapting Programmatic LM Workflows to Data
Chu-Cheng Lin, Daiyi Peng, Yifeng Lu +2
Reliably composing Large Language Models (LLMs) for complex, multi-step workflows remains a significant challenge. The dominant paradigm -- optimizing discrete prompts in a pipelin…
Reasoning-Finetuning Repurposes Latent Representations in Base Models
Jake Ward, Chuqiao Lin, Constantin Venhoff +1
Backtracking, an emergent behavior elicited by reasoning fine-tuning, has been shown to be a key mechanism in reasoning models' enhanced capabilities. Prior work has succeeded in m…
Inducing Generalization across Languages and Tasks using Featurized Low-Rank Mixtures
Chu-Cheng Lin, Xinyi Wang, Jonathan H. Clark +4
Adapting pretrained large language models (LLMs) to various downstream tasks in tens or hundreds of human languages is computationally expensive. Parameter-efficient fine-tuning (P…
Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection
Yun Zhu, Jia-Chen Gu, Caitlin Sikora +8
Large language models (LLMs) augmented with retrieval exhibit robust performance and extensive versatility by incorporating external contexts. However, the input length grows linea…
Low-Rank Adaptation for Multilingual Summarization: An Empirical Study
Chenxi Whitehouse, Fantine Huot, Jasmijn Bastings +3
Although the advancements of pre-trained Large Language Models have significantly accelerated recent progress in NLP, their ever-increasing size poses significant challenges for co…