3 papers
cs.LG2025
BigBang-Proton Technical Report: Next-Word-Prediction is Scientific Multitask Learner
Hengkui Wu, Liujiang Liu, Jihua He +23
We introduce BigBang-Proton, a unified sequence-based architecture for auto-regressive language modeling pretrained on cross-scale, cross-structure, cross-discipline real-world sci…
cs.LG2024
Scaling Particle Collision Data Analysis
Hengkui Wu, Panpan Chi, Yongfeng Zhu +10
For decades, researchers have developed task-specific models to address scientific challenges across diverse disciplines. Recently, large language models (LLMs) have shown enormous…
cs.CL2023
Source Prompt: Coordinated Pre-training of Language Models on Diverse Corpora from Multiple Sources
Yipei Xu, Dakuan Lu, Jiaqing Liang +7
Pre-trained language models (PLMs) have established the new paradigm in the field of NLP. For more powerful PLMs, one of the most popular and successful way is to continuously scal…