20 citations · 102 across the 20 of their papers we have counts for
4 papers · 1 filter
How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning?
Yang Luo, Zangwei Zheng, Zirui Zhu +1
The increase in parameter size of multimodal large language models (MLLMs) introduces significant capabilities, particularly in-context learning, where MLLMs enhance task performan…
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
Fuzhao Xue, Zian Zheng, Yao Fu +4
To help the open-source community have a better understanding of Mixture-of-Experts (MoE) based large language models (LLMs), we train and release OpenMoE, a series of fully open-s…
CAME: Confidence-guided Adaptive Memory Efficient Optimization
Yang Luo, Xiaozhe Ren, Zangwei Zheng +3
Adaptive gradient methods, such as Adam and LAMB, have demonstrated excellent performance in the training of large language models. Nevertheless, the need for adaptivity requires m…
Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline
Zangwei Zheng, Xiaozhe Ren, Fuzhao Xue +3
Large language models (LLMs) have revolutionized the field of AI, demonstrating unprecedented capacity across various tasks. However, the inference process for LLMs comes with sign…