12 citations · 15 across the 12 of their papers we have counts for
5 papers · 1 filter
ReScraper: Unified Scraping and Cleaning of Web Data for Effective LLM Pretraining
Zichun Yu, Jiarui Yan, Shlok Sanghvi +2
LLM pretraining corpora are normally cleaned by a stack of hand-written heuristics. A heuristic scraper extracts the main content from HTML, and dozens of rule-based filters then c…
Spectral Guidance for Flexible and Efficient Control of Diffusion Models
Gabriel Moreira, Manuel Marques, João Paulo Costeira +1
We introduce Spectral Guidance, a framework for controlling diffusion models by leveraging the intrinsic geometry of the generative process. As data is progressively corrupted by n…
Generating Pretraining Tokens from Organic Data for Data-Bound Scaling
Zichun Yu, Chenyan Xiong
LLM pretraining is shifting from a compute-bound to a data-bound regime, where available human (organic) text falls far short of scaling demands. However, reaching the data-bound r…
SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks
Shanshan Zhong, Yi Lu, Jingjie Ning +7
Skills have become the de facto way to enable LLM agents to perform complex real-world tasks with customized instructions, workflows, and tools, but how to learn them automatically…
EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training
Yiyang Du, Zhanqiu Guo, Haochen Zhang +5
Vision-Language-Action Models (VLAs) inherit their visual and linguistic capabilities from Vision-Language Models (VLMs), yet most VLAs are built from off-the-shelf VLMs that are n…