Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Beyond a Single Extractor: Re-thinking HTML-to-Text Extraction for LLM Pretraining
Jeffrey Li, Josh Gardner, Doug Kang +10
One of the first pre-processing steps for constructing web-scale LLM pretraining datasets involves extracting text from HTML. Despite the immense diversity of web content, existing…
cs.CL2026
Relative Scaling Laws for LLMs
William Held, David Hall, Percy Liang +1
Scaling laws describe how language models improve with additional data, parameters, and compute. While widely used, they are typically measured on aggregate test sets. Aggregate ev…