12 papers
A Target-Centric Survey of Quantization-Aware Training
Jiamin Song, Mengjie Zhao, Zijing Wang +6
The rapid development of LLMs incurs prohibitive memory footprints and intensive computational demands. Quantization-Aware Training (QAT) techniques have emerged as a promising sol…
TRAM: Enhancing Multimodal Reasoning with Trajectory-Derived Auxiliary Memory
Kang Liu, Zijing Wang, Yongkang Liu +5
Multimodal Large Reasoning Models (MLRMs) have achieved strong performance on tasks requiring visual understanding and multi-step inference. However, as reasoning trajectories grow…
Deformba: Vision State Space Model with Adaptive State Fusion
Hongyu Ke, Jack Morris, Yongkang Liu +4
State Space Models (SSMs) have emerged as a powerful and efficient alternative to Transformers, demonstrating linear-time complexity and exceptional sequence modeling capabilities.…
ChunkFT: Byte-Streamed Optimization for Memory-Efficient Full Fine-Tuning
Yongkang Liu, Zijing Wang, Mengjie Zhao +7
This work presents \textsc{ChunkFT}, a memory-efficient fine-tuning framework that reformulates full-parameter fine-tuning around a dynamically activated working set. \textsc{Chunk…
SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
Yongkang Liu, Xing Li, Mengjie Zhao +7
As the number of model parameters increases, parameter-efficient fine-tuning (PEFT) has become the go-to choice for tailoring pre-trained large language models. Low-rank Adaptation…
DiM\textsuperscript{3}: Bridging Multilingual and Multimodal Models via Direction- and Magnitude-Aware Merging
Zijing Wang, Mingyang Wang, Ercong Nie +6
Towards more general and human-like intelligence, large language models should seamlessly integrate both multilingual and multimodal capabilities; however, extending an existing mu…
A Systematic Analysis of the Impact of Persona Steering on LLM Capabilities
Jiaqi Chen, Ming Wang, Tingna Xie +2
Imbuing Large Language Models (LLMs) with specific personas is prevalent for tailoring interaction styles, yet the impact on underlying cognitive capabilities remains unexplored. W…
How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A
YiJie Huang, Yiqun Zhang, Zhuoyue Jia +7
Vision-language models improve perception by feeding increasingly long visual token sequences into language backbones, but the resulting inference cost raises a basic scaling quest…
From Parameter Dynamics to Risk Scoring : Quantifying Sample-Level Safety Degradation in LLM Fine-tuning
Xiao Wang, Yifei Zhang, YongKang Liu +4
Safety alignment of Large Language Models (LLMs) is extremely fragile, as fine-tuning on a small number of benign samples can erase safety behaviors learned from millions of prefer…
Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Model
Ling Team, Anqi Shen, Baihui Li +101
We present Ring-1T, the first open-source, state-of-the-art thinking model with a trillion-scale parameter. It features 1 trillion total parameters and activates approximately 50 b…
Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs
Ling Team, Bin Hu, Cai Chen +43
We present Ring-lite, a Mixture-of-Experts (MoE)-based large language model optimized via reinforcement learning (RL) to achieve efficient and robust reasoning capabilities. Built…
Holistic Capability Preservation: Towards Compact Yet Comprehensive Reasoning Models
Ling Team, Caizhi Tang, Chilin Fu +15
This technical report presents Ring-Lite-Distill, a lightweight reasoning model derived from our open-source Mixture-of-Experts (MoE) Large Language Models (LLMs) Ling-Lite. This s…