4 papers
Curriculum Learning for LLM Pretraining: An Analysis of Learning Dynamics
Mohamed Elgaar, Hadi Amiri
Curriculum learning changes the order of pretraining data, but it remains unclear how ordering changes the learning dynamics. We pretrain models from 14M to 1B parameters for 300B…
Linguistic Blind Spots in Clinical Decision Extraction
Mohamed Elgaar, Hadi Amiri
Extracting medical decisions from clinical notes is a key step for clinical decision support and patient-facing care summaries. We study how the linguistic characteristics of clini…
LingGen: Scalable Multi-Attribute Linguistic Control via Power-Law Masking
Mohamed Elgaar, Hadi Amiri
We present LingGen, a controlled text generation model that allows fine-grained control over a large number of real-valued linguistic attributes. It encodes target attribute values…
Linguistically-Controlled Paraphrase Generation
Mohamed Elgaar, Hadi Amiri
Controlled paraphrase generation produces paraphrases that preserve meaning while allowing precise control over linguistic attributes of the output. We introduce LingConv, an encod…