most citedPrE-Text: Training Language Models on Private Federated Data in the Age of LLMs

1 citations · 1 across the 4 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2026

Short Data, Long Context: Distilling Positional Knowledge in Transformers

Patrick Huber, Ernie Chang, Chinnadhurai Sankar +4

Extending the context window of language models typically requires expensive long-context pre-training, posing significant challenges for both training efficiency and data collecti…

cs.CL2024

CoDi: Conversational Distillation for Grounded Question Answering

Patrick Huber, Arash Einolghozati, Rylan Conway +6

Distilling conversational skills into Small Language Models (SLMs) with approximately 1 billion parameters presents significant challenges. Firstly, SLMs have limited capacity in t…