12 citations · 39 across the 12 of their papers we have counts for
12 papers
SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD
Dongfang Li, Xiaodong Luo, Ruoyu Sun +62
Full-parameter post-training of trillion-parameter-scale MoE models introduces substantial system-level challenges for large-scale distributed training, including severe memory pre…
EDCO: Dynamic Curriculum Orchestration for Domain-specific Large Language Model Fine-tuning
Jing-Cheng Pang, Liu Sun, Chang Zhou +10
Domain-specific large language models (LLMs), typically developed by fine-tuning a pre-trained general-purpose LLM on specialized datasets, represent a significant advancement in a…
Adam-mini: Use Fewer Learning Rates To Gain More
Yushun Zhang, Congliang Chen, Ziniu Li +6
We propose Adam-mini, an optimizer that achieves on par or better performance than AdamW with 50% less memory footprint. Adam-mini reduces memory by cutting down the learning rate…
The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Models
Chenwei Wu, Li Erran Li, Stefano Ermon +3
Compositionality is a common property in many modalities including natural languages and images, but the compositional generalization of multi-modal models is not well-understood.…
BenchMD: A Benchmark for Unified Learning on Medical Images and Sensors
Kathryn Wantlin, Chenwei Wu, Shih-Cheng Huang +12
Medical data poses a daunting challenge for AI algorithms: it exists in many different modalities, experiences frequent distribution shifts, and suffers from a scarcity of examples…
Hiding Data Helps: On the Benefits of Masking for Sparse Coding
Muthu Chidambaram, Chenwei Wu, Yu Cheng +1
Sparse coding, which refers to modeling a signal as sparse linear combinations of the elements of a learned dictionary, has proven to be a successful (and interpretable) approach i…