7 papers
TriSpec: Ternary Speculative Decoding via Lightweight Proxy Verification
Haoyun Jiang, Junqi He, Feng Hong +8
Inference efficiency in Large Language Models (LLMs) is fundamentally limited by their serial, autoregressive generation, especially as reasoning becomes a key capability and respo…
Dual-granularity Sinkhorn Distillation for Enhanced Learning from Long-tailed Noisy Data
Feng Hong, Yu Huang, Zihua Zhao +5
Real-world datasets for deep learning frequently suffer from the co-occurring challenges of class imbalance and label noise, hindering model performance. While methods exist for ea…
Long-tailed Recognition with Model Rebalancing
Jiaan Luo, Feng Hong, Qiang Hu +3
Long-tailed recognition is ubiquitous and challenging in deep learning and even in the downstream finetuning of foundation models, since the skew class distribution generally preve…
Differential-informed Sample Selection Accelerates Multimodal Contrastive Learning
Zihua Zhao, Feng Hong, Mengxi Chen +5
The remarkable success of contrastive-learning-based multimodal models has been greatly driven by training on ever-larger datasets with expensive compute consumption. Sample select…
Innovator: Scientific Continued Pretraining with Fine-grained MoE Upcycling
Ning Liao, Xiaoxing Wang, Zehao Lin +18
A large language model (LLM) with knowledge in both scientific and general tasks is the foundation of science general intelligence. However, directly continued pretraining an LLM u…
Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs
Feng Hong, Geng Yu, Yushi Ye +5
Diffusion Large Language Models (DLLMs) have emerged as a compelling alternative to Autoregressive models, designed for fast parallel generation. However, existing DLLMs are plague…