18 papers
Stable FP4 Training via Transposition-Invariant Block Quantization
Mehdi Rahimifar, Amin Darabi, Mehran Taghian Jazi +6
Reducing training precision is a key lever for improving the e ciency of large language model (LLM) training, but pushing beyond FP8 to 4-bit oating point (FP4) remains challenging…
FinSAgent: Corpus-Aligned Multi-Agent RAG Framework for Evidence-Grounded SEC Filing Question Answering
Jijun Chi, Zhenghan Tai, Hanwei Wu +21
Financial question answering over U.S. Securities and Exchange Commission (SEC) filings requires retrieving and synthesizing heterogeneous evidence dispersed across long, standardi…
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Maxime Heuillet, Yufei Cui, Boxing Chen +2
Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT). In standard ReFT framewor…
MATCH: Modulating Attention via In-Context Retrieval for Long-Context Transformers
Linrui Ma, Chun Hei Lo, Xinyu Wang +12
The quadratic computational cost of traditional attention mechanisms poses a major bottleneck to the scalability and practical deployment of large language models (LLMs), particula…
HoloQ-VLA: Uniform W4A4 Quantization of Vision-Language-Action Models
Xinyu Wang, Mingze Li, Sicheng Lyu +6
Vision-Language-Action (VLA) models unify perception, reasoning, and control in a single policy, but their multi-billion-parameter backbones and diffusion-based action heads make o…
Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing
Xinyu Wang, Sicheng Lyu, Yu Gu +4
Model editing updates a pre-trained LLM with new facts or rules without retraining while preserving unrelated behavior. In real deployment, edits arrive as long streams, creating a…