5 citations · 15 across the 32 of their papers we have counts for
14 papers · 1 filter
PT-LLM: Post-Training Ternarization for Large Language Models
Xianglong Yan, Chengzhu Bao, Zhiteng Li +6
Large Language Models (LLMs) have shown impressive capabilities across diverse tasks, but their large memory and compute demands hinder deployment. Ternarization has gained attenti…
Towards a Comprehensive Scaling Law of Mixture-of-Experts
Guoliang Zhao, Yuhan Fu, Shuaipeng Li +10
Mixture-of-Experts (MoE) models have become the consensus approach for enabling parameter-efficient scaling and cost-effective deployment in large language models. However, existin…
Hybrid-Tower: Fine-grained Pseudo-query Interaction and Generation for Text-to-Video Retrieval
Bangxiang Lan, Ruobing Xie, Ruixiang Zhao +4
The Text-to-Video Retrieval (T2VR) task aims to retrieve unlabeled videos by textual queries with the same semantic meanings. Recent CLIP-based approaches have explored two framewo…
Proximal Supervised Fine-Tuning
Wenhong Zhu, Ruobing Xie, Rui Wang +3
Supervised fine-tuning (SFT) of foundation models often leads to poor generalization, where prior capabilities deteriorate after tuning on new tasks or domains. Inspired by trust-r…
Fighting Fire with Fire (F3): A Training-free and Efficient Visual Adversarial Example Purification Method in LVLMs
Yudong Zhang, Ruobing Xie, Yiqing Huang +5
Recent advances in large vision-language models (LVLMs) have showcased their remarkable capabilities across a wide range of multimodal vision-language tasks. However, these models…
The Security Threat of Compressed Projectors in Large Vision-Language Models
Yudong Zhang, Ruobing Xie, Xingwu Sun +4
The choice of a suitable visual language projector (VLP) is critical to the successful training of large visual language models (LVLMs). Mainstream VLPs can be broadly categorized…
The Climb Carves Wisdom Deeper Than the Summit: On the Noisy Rewards in Learning to Reason
Ang Lv, Ruobing Xie, Xingwu Sun +2
Recent studies on post-training large language models (LLMs) for reasoning through reinforcement learning (RL) typically focus on tasks that can be accurately verified and rewarded…
Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought
Tencent Hunyuan Team, Ao Liu, Botong Zhou +248
As Large Language Models (LLMs) rapidly advance, we introduce Hunyuan-TurboS, a novel large hybrid Transformer-Mamba Mixture of Experts (MoE) model. It synergistically combines Mam…
QAVA: Query-Agnostic Visual Attack to Large Vision-Language Models
Yudong Zhang, Ruobing Xie, Jiansheng Chen +3
In typical multimodal tasks, such as Visual Question Answering (VQA), adversarial attacks targeting a specific image and question can lead large vision-language models (LVLMs) to p…
Large Language Model Empowered Recommendation Meets All-domain Continual Pre-Training
Haokai Ma, Yunshan Ma, Ruobing Xie +5
Recent research efforts have investigated how to integrate Large Language Models (LLMs) into recommendation, capitalizing on their semantic comprehension and open-world knowledge f…
TransMamba: A Sequence-Level Hybrid Transformer-Mamba Language Model
Yixing Li, Ruobing Xie, Zhen Yang +8
Transformers are the cornerstone of modern large language models, but their quadratic computational complexity limits efficiency in long-sequence processing. Recent advancements in…
Multi-Grained Patch Training for Efficient LLM-based Recommendation
Jiayi Liao, Ruobing Xie, Sihang Li +4
Large Language Models (LLMs) have emerged as a new paradigm for recommendation by converting interacted item history into language modeling. However, constrained by the limited con…
Autonomy-of-Experts Models
Ang Lv, Ruobing Xie, Yining Qian +5
Mixture-of-Experts (MoE) models mostly use a router to assign tokens to specific expert modules, activating only partial parameters and often outperforming dense models. We argue t…
Scaling Laws for Floating Point Quantization Training
Xingwu Sun, Shuaipeng Li, Ruobing Xie +13
Low-precision training is considered an effective strategy for reducing both training and downstream inference costs. Previous scaling laws for precision mainly focus on integer qu…