3 citations · 4 across the 8 of their papers we have counts for
8 papers
Scalable Frequency- and Length-Aware Subdocument Deduplication for Large Language Model Pretraining
Hai Wang, Chenhao Wang, Qifeng Cai +6
Large-scale pretraining corpora contain substantial duplicate content. Although document-level deduplication is widely used, removing subdocument-level redundancy remains challengi…
Hy-MT2: A Family of Fast, Efficient and Powerful Multilingual Translation Models in the Wild
Mao Zheng, Zheng Li, Tao Chen +10
Hy-MT2 is a family of fast-thinking multilingual translation models designed for complex real-world scenarios. It includes three model sizes: 1.8B, 7B, and 30B-A3B (MoE), all of wh…
Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought
Tencent Hunyuan Team, Ao Liu, Botong Zhou +248
As Large Language Models (LLMs) rapidly advance, we introduce Hunyuan-TurboS, a novel large hybrid Transformer-Mamba Mixture of Experts (MoE) model. It synergistically combines Mam…
LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning
Jiachun Li, Pengfei Cao, Chenhao Wang +6
Large language models (LLMs) sometimes demonstrate poor performance on knowledge-intensive tasks, commonsense reasoning is one of them. Researchers typically address these issues b…
RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
Zhuoran Jin, Pengfei Cao, Chenhao Wang +6
Large language models (LLMs) inevitably memorize sensitive, copyrighted, and harmful knowledge from the training corpus; therefore, it is crucial to erase this knowledge from the m…
AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation
Zhitao He, Pengfei Cao, Chenhao Wang +7
With the development of deep learning, natural language processing technology has effectively improved the efficiency of various aspects of the traditional judicial industry. Howev…
Focus on Your Question! Interpreting and Mitigating Toxic CoT Problems in Commonsense Reasoning
Jiachun Li, Pengfei Cao, Chenhao Wang +5
Large language models exhibit high-level commonsense reasoning abilities, especially with enhancement methods like Chain-of-Thought (CoT). However, we find these CoT-like methods l…
CogNet: Bridging Linguistic Knowledge, World Knowledge and Commonsense Knowledge
Chenhao Wang, Yubo Chen, Zhipeng Xue +2
In this paper, we present CogNet, a knowledge base (KB) dedicated to integrating three types of knowledge: (1) linguistic knowledge from FrameNet, which schematically describes sit…