2 citations · 4 across the 12 of their papers we have counts for
4 papers · 1 filter
TopoCompress: Long Context Compression via Graph-Wired Semantic Trajectories
Daniel Agyei Asante, Yang Li
Long-context compression is essential for reducing the cost and latency of large language model inference. However, existing methods can fragment important evidence, require additi…
AutoMixer: Checkpoint Artifacts as Automatic Data Mixers
Ernie Chang, Yang Li, Patrick Huber +4
In language model training, it is desirable to equip models with capabilities from various tasks. However, it is not clear how to directly obtain the right data mixtures for these…
Scaling Parameter-Constrained Language Models with Quality Data
Ernie Chang, Matteo Paltenghi, Yang Li +7
Scaling laws in language modeling traditionally quantify training loss as a function of dataset size and model parameters, providing compute-optimal estimates but often neglecting…
Target-Aware Language Modeling via Granular Data Sampling
Ernie Chang, Pin-Jie Lin, Yang Li +6
Language model pretraining generally targets a broad range of use cases and incorporates data from diverse sources. However, there are instances where we desire a model that excels…