3 papers
cs.CL2024
Scaling Parameter-Constrained Language Models with Quality Data
Ernie Chang, Matteo Paltenghi, Yang Li +7
Scaling laws in language modeling traditionally quantify training loss as a function of dataset size and model parameters, providing compute-optimal estimates but often neglecting…
cs.CL2024
Target-Aware Language Modeling via Granular Data Sampling
Ernie Chang, Pin-Jie Lin, Yang Li +6
Language model pretraining generally targets a broad range of use cases and incorporates data from diverse sources. However, there are instances where we desire a model that excels…
cs.DS2016
Using Sequence Ensembles for Seeding Alignments of MinION Sequencing Data
Rastislav Rabatin, Broňa Brejová, Tomáš Vinař
Oxford Nanopore MinION sequencer is currently the smallest sequencing device available. While being able to produce very long reads (reads of up to 100~kbp were reported), it is pr…