4 papers
Yuan3.0 Ultra: A Trillion-Parameter Enterprise-Oriented MoE LLM
YuanLab. ai, :, Shawn Wu +25
We introduce Yuan3.0 Ultra, an open-source Mixture-of-Experts (MoE) large language model featuring 68.8B activated parameters and 1010B total parameters, specially designed to enha…
Yuan3.0 Flash: An Open Multimodal Large Language Model for Enterprise Applications
YuanLab. ai, :, Shawn Wu +24
We introduce Yuan3.0 Flash, an open-source Mixture-of-Experts (MoE) MultiModal Large Language Model featuring 3.7B activated parameters and 40B total parameters, specifically desig…
Yuan-TecSwin: A text conditioned Diffusion model with Swin-transformer blocks
Shaohua Wu, Tong Yu, Shenling Wang +1
Diffusion models have shown remarkable capacity in image synthesis based on their U-shaped architecture and convolutional neural networks (CNN) as basic blocks. The locality of the…
PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs
Tongzhou Yu, Zhuhao Zhang, Guanghui Zhu +3
Parameter Efficient Fine-Tuning (PEFT) methods have emerged as effective and promising approaches for fine-tuning pre-trained language models. Compared with Full parameter Fine-Tun…