4 citations · 11 across the 12 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2024
Scaling Law Hypothesis for Multimodal Model
Qingyun Sun, Zhen Guo, PIN AI Team
We propose a scaling law hypothesis for multimodal models processing text, audio, images, and video within a shared token and embedding space. Our framework predicts model performa…
cs.LG2024
More Compute Is What You Need
Zhen Guo
Large language model pre-training has become increasingly expensive, with most practitioners relying on scaling laws to allocate compute budgets for model size and training tokens,…
cs.LG2024
Diversity Measurement and Subset Selection for Instruction Tuning Datasets
Peiqi Wang, Yikang Shen, Zhen Guo +4
We aim to select data subsets for the fine-tuning of large language models to more effectively follow instructions. Prior work has emphasized the importance of diversity in dataset…