11 papers
TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers
Peng Cheng, Jiucheng Zang, Qingnan Li +6
Muon-style optimizers leverage Newton-Schulz (NS) iterations to orthogonalize updates, yielding update geometries that often outperform Adam-series methods. However, this orthogona…
ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
Haley Li, Xinglu Wang, Cong Feng +12
As LLM deployments scale over more hardware, the probability of a single failure in a system increases significantly, and cloud operators must consider robust countermeasures to ha…
Thinking Long, but Short: Stable Sequential Test-Time Scaling for Large Reasoning Models
Michael R. Metel, Yufei Cui, Boxing Chen +1
Sequential test-time scaling is a promising training-free method to improve large reasoning model accuracy, but as currently implemented, significant limitations have been observed…
: A Route-to-Rerank Post-Training Framework for Multi-Domain Decoder-Only Rerankers
Xinyu Wang, Hanwei Wu, Qingchen Hu +13
Decoder-only rerankers are central to Retrieval-Augmented Generation (RAG). However, generalist models miss domain-specific nuances in high-stakes fields like finance and law, and…
VeritasFi: An Adaptable, Multi-tiered RAG Framework for Multi-modal Financial Question Answering
Zhenghan Tai, Hanwei Wu, Qingchen Hu +24
Retrieval-Augmented Generation (RAG) is becoming increasingly essential for Question Answering (QA) in the financial sector, where accurate and contextually grounded insights from…
EvoEdit: Evolving Null-space Alignment for Robust and Efficient Knowledge Editing
Sicheng Lyu, Yu Gu, Xinyu Wang +5
Large language models (LLMs) require continual updates to rectify outdated or erroneous knowledge. Model editing has emerged as a compelling paradigm for introducing targeted modif…