4 citations · 11 across the 12 of their papers we have counts for
6 papers · 1 filter
Scaling Law Hypothesis for Multimodal Model
Qingyun Sun, Zhen Guo, PIN AI Team
We propose a scaling law hypothesis for multimodal models processing text, audio, images, and video within a shared token and embedding space. Our framework predicts model performa…
Octo-planner: On-device Language Model for Planner-Action Agents
Wei Chen, Zhiyuan Li, Zhen Guo +1
AI agents have become increasingly significant in various domains, enabling autonomous decision-making and problem-solving. To function effectively, these agents require a planning…
More Compute Is What You Need
Zhen Guo
Large language model pre-training has become increasingly expensive, with most practitioners relying on scaling laws to allocate compute budgets for model size and training tokens,…
JetMoE: Reaching Llama2 Performance with 0.1M Dollars
Yikang Shen, Zhen Guo, Tianle Cai +1
Large Language Models (LLMs) have achieved remarkable results, but their increasing resource demand has become a major obstacle to the development of powerful and accessible super-…
Diversity Measurement and Subset Selection for Instruction Tuning Datasets
Peiqi Wang, Yikang Shen, Zhen Guo +4
We aim to select data subsets for the fine-tuning of large language models to more effectively follow instructions. Prior work has emphasized the importance of diversity in dataset…
API Pack: A Massive Multi-Programming Language Dataset for API Call Generation
Zhen Guo, Adriana Meza Soria, Wei Sun +2
We introduce API Pack, a massive multi-programming language dataset containing over one million instruction-API calls for improving the API call generation capabilities of large la…