8 papers
Scalable Frequency- and Length-Aware Subdocument Deduplication for Large Language Model Pretraining
Hai Wang, Chenhao Wang, Qifeng Cai +6
Large-scale pretraining corpora contain substantial duplicate content. Although document-level deduplication is widely used, removing subdocument-level redundancy remains challengi…
DataPrep-Bench: Benchmarking LLMs as Training Data Preparators
Hao Liang, Qifeng Cai, Yibo Lin +11
The quality of training data fundamentally determines the capabilities of large language models (LLMs), yet no unified benchmark exists to measure how well LLMs, agents, and data-c…
DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models
Hao Liang, Zhengyang Zhao, Meiyi Qiang +22
Data-centric training has emerged as a promising direction for improving large language models (LLMs) by optimizing not only model parameters but also the selection, composition, a…
Towards Next-Generation LLM Training: From the Data-Centric Perspective
Hao Liang, Zhengyang Zhao, Zhaoyang Han +8
Large language models (LLMs) have demonstrated remarkable performance across a wide range of tasks and domains, with data playing a central role in enabling these advances. Despite…
Text2SQL-Flow: A Robust SQL-Aware Data Augmentation Framework for Text-to-SQL
Qifeng Cai, Hao Liang, Chang Xu +3
The data-centric paradigm has emerged as a pivotal direction in artificial intelligence (AI), emphasizing the role of high-quality training data. This shift is especially critical…
LoVR: A Benchmark for Long Video Retrieval in Multimodal Contexts
Qifeng Cai, Hao Liang, Zhaoyang Han +6
Long videos contain a vast amount of information, making video-text retrieval an essential and challenging task in multimodal learning. However, existing benchmarks suffer from lim…