1 citations · 1 across the 9 of their papers we have counts for
6 papers · 1 filter
Scaling Embeddings Outperforms Scaling Experts in Language Models
Hong Liu, Jiaqi Zhang, Chao Wang +13
While Mixture-of-Experts (MoE) architectures have become the standard for sparsity scaling in large language models, they increasingly face diminishing returns and system-level bot…
Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text
Zhihao Xu, Rumei Li, Jiahuan Li +4
Enabling Large Language Models (LLMs) to effectively utilize tools in multi-turn interactions is essential for building capable autonomous agents. However, acquiring diverse and re…
A Survey on LLM Mid-Training
Chengying Tu, Xuemiao Zhang, Rongxiang Weng +6
Recent advances in foundation models have highlighted the significant benefits of multi-stage training, with a particular emphasis on the emergence of mid-training as a vital stage…
LongCat-Flash Technical Report
Meituan LongCat Team, Bayan, Bei Li +179
We introduce LongCat-Flash, a 560-billion-parameter Mixture-of-Experts (MoE) language model designed for both computational efficiency and advanced agentic capabilities. Stemming f…
SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models
Muxi Diao, Rumei Li, Shiyang Liu +4
As large language models (LLMs) continue to advance in capability and influence, ensuring their security and preventing harmful outputs has become crucial. A promising approach to…
Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQA
Guanting Dong, Rumei Li, Sirui Wang +3
Knowledge Base Question Answering (KBQA) aims to answer natural language questions with factual information such as entities and relations in KBs. However, traditional Pre-trained…