From the 1 of 4 linked papers with an AI index.
4 papers
SkillSmith: Learning to Compose Parametric Skills and Textual Knowledge
Lucio M. Dery, Benedict Aaron Tjandra, Siavash Samiei +4
SkillSmith is a method that lets a large language model reason over both textual knowledge and prefix‑tuned model weights, enabling it to generate new parametric skill prefixes for…
Context Training with Active Information Seeking
Zeyu Huang, Adhiguna Kuncoro, Qixuan Feng +4
Most existing large language models (LLMs) are expensive to adapt after deployment, especially when a task requires newly produced information or niche domain knowledge. Recent wor…
Latent Space Communication via K-V Cache Alignment
Lucio M. Dery, Zohar Yahav, Henry Prior +3
Solving increasingly complex problems with large language models (LLMs) necessitates a move beyond individual models and towards multi-model systems that can effectively collaborat…
Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch
Arthur Douillard, Yanislav Donchev, Keith Rush +11
Training of large language models (LLMs) is typically distributed across a large number of accelerators to reduce training time. Since internal states and parameter gradients need…