3 citations · 6 across the 6 of their papers we have counts for
6 papers
HRM-Text: Efficient Pretraining Beyond Scaling
Guan Wang, Changling Liu, Chenyu Wang +6
The current pretraining paradigm for large language models relies on massive compute and internet-scale raw text, creating a significant barrier to foundational research. In contra…
Why Attend to Everything? Focus is the Key
Hengshuai Yao, Xing Chen, Ahmed Murtadha +8
Standard attention scales quadratically with sequence length. Efficient attention methods reduce this O(n^2) cost, but when retrofitted into pretrained models, they often degrade p…
Hierarchical Reasoning Model
Guan Wang, Jin Li, Yuhao Sun +6
Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language models (LLMs) primarily employ C…
Breeze-7B Technical Report
Chan-Jan Hsu, Chang-Le Liu, Feng-Ting Liao +3
Breeze-7B is an open-source language model based on Mistral-7B, designed to address the need for improved language comprehension and chatbot-oriented capabilities in Traditional Ch…
Advancing the Evaluation of Traditional Chinese Language Models: Towards a Comprehensive Benchmark Suite
Chan-Jan Hsu, Chang-Le Liu, Feng-Ting Liao +3
The evaluation of large language models is an essential task in the field of language understanding and generation. As language models continue to advance, the need for effective b…
Extending the Pre-Training of BLOOM for Improved Support of Traditional Chinese: Models, Methods and Results
Philipp Ennen, Po-Chun Hsu, Chan-Jan Hsu +6
In this paper we present the multilingual language model BLOOM-zh that features enhanced support for Traditional Chinese. BLOOM-zh has its origins in the open-source BLOOM models p…