2 citations · 2 across the 4 of their papers we have counts for
5 papers
More Than Seeing: Vision as Scalable Anchors in Native Multimodal Pre-Training
Haoyuan Wu, Aoqi Wu, Jiani Hu +1
Native multimodal pre-training trains a single model from scratch on text and multimodal data, enabling it to understand both language and visual inputs. However, how visual signal…
Scaling Native Multimodal Pre-Training From Scratch
Haoyuan Wu, Aoqi Wu, Hai Wang +3
Although large language models (LLMs) exhibit remarkable reasoning capabilities, their reliance on text-only pre-training restricts the perception of the multimodal physical world.…
Diversity or Precision? A Deep Dive into Next Token Prediction
Haoyuan Wu, Hai Wang, Jiajia Wu +5
Recent advancements have shown that reinforcement learning (RL) can substantially improve the reasoning abilities of large language models (LLMs). The effectiveness of such RL trai…
Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
Haoyuan Wu, Haoxing Chen, Xiaodong Chen +10
The Mixture of Experts (MoE) architecture is a cornerstone of modern state-of-the-art (SOTA) large language models (LLMs). MoE models facilitate scalability by enabling sparse para…
TinyFormer: Efficient Transformer Design and Deployment on Tiny Devices
Jianlei Yang, Jiacheng Liao, Fanding Lei +6
Developing deep learning models on tiny devices (e.g. Microcontroller units, MCUs) has attracted much attention in various embedded IoT applications. However, it is challenging to…