Publications (26)
Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aakshita Chandiramani +544
We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemo…
UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models
Huawei Lin, Yingjie Lao, Tong Geng +2
Large Language Models (LLMs) are vulnerable to attacks like prompt injection, backdoor attacks, and adversarial attacks, which manipulate prompts or models to generate harmful outp…
Degenerate Swin to Win: Plain Window-based Transformer without Sophisticated Operations
Tan Yu, Ping Li
The formidable accomplishment of Transformers in natural language processing has motivated the researchers in the computer vision community to build Vision Transformers. Compared w…
S-MLP: Spatial-Shift MLP Architecture for Vision
Tan Yu, Xu Li, Yunfeng Cai +2
Recently, visual Transformer (ViT) and its following works abandon the convolution and exploit the self-attention operation, attaining a comparable or even higher accuracy than CNN…
Beyond the limitation of a single query: Train your LLM for query expansion with Reinforcement Learning
Shu Zhao, Tan Yu, Anbang Xu
Reasoning-augmented search agents, such as Search-R1, are trained to reason, search, and generate the final answer iteratively. Nevertheless, due to their limited capabilities in r…
Marrying Autoregressive Transformer and Diffusion with Multi-Reference Autoregression
Dingcheng Zhen, Qian Qiao, Xu Zheng +6
We introduce TransDiff, the first image generation model that marries Autoregressive (AR) Transformer with diffusion models. In this joint modeling framework, TransDiff encodes lab…
SoulX-FlashHead: Oracle-guided Generation of Infinite Real-time Streaming Talking Heads
Tan Yu, Qian Qiao, Le Shen +9
Achieving a balance between high-fidelity visual quality and low-latency streaming remains a formidable challenge in audio-driven portrait generation. Existing large-scale models o…
ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning
Shu Zhao, Tan Yu, Anbang Xu +3
Reasoning-augmented search agents such as Search-R1, trained via reinforcement learning with verifiable rewards (RLVR), demonstrate remarkable capabilities in multi-step informatio…
Adaptive Data Flywheel: Applying MAPE Control Loops to AI Agent Improvement
Aaditya Shukla, Sidney Knowles, Meenakshi Madugula +9
Enterprise AI agents must continuously adapt to maintain accuracy, reduce latency, and remain aligned with user needs. We present a practical implementation of a data flywheel in N…
Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models
Yue Zhang, Hongliang Fei, Dingcheng Li +2
Prompt learning is a new learning paradigm which reformulates downstream tasks as similar pretraining tasks on pretrained models by leveraging textual prompts. Recent works have de…
Tree-based Text-Vision BERT for Video Search in Baidu Video Advertising
Tan Yu, Jie Liu, Yi Yang +3
The advancement of the communication technology and the popularity of the smart phones foster the booming of video ads. Baidu, as one of the leading search engine companies in the…
R2-MLP: Round-Roll MLP for Multi-View 3D Object Recognition
Shuo Chen, Tan Yu, Ping Li
Recently, vision architectures based exclusively on multi-layer perceptrons (MLPs) have gained much attention in the computer vision community. MLP-like models achieve competitive…
Reconstruct before Query: Continual Missing Modality Learning with Decomposed Prompt Collaboration
Shu Zhao, Xiaohan Zou, Tan Yu +1
Pre-trained large multi-modal models (LMMs) exploit fine-tuning to adapt diverse user applications. Nevertheless, fine-tuning may face challenges due to deactivated sensors (e.g.,…
RAP: Real-time Audio-driven Portrait Animation with Video Diffusion Transformer
Fangyu Du, Taiqing Li, Qian Qiao +7
Audio-driven portrait animation aims to synthesize realistic and natural talking head videos from an input audio signal and a single reference image. While existing methods achieve…
SoulX-FlashTalk: Real-Time Infinite Streaming of Audio-Driven Avatars via Self-Correcting Bidirectional Distillation
Le Shen, Qian Qiao, Tan Yu +7
Deploying massive diffusion models for real-time, infinite-duration, audio-driven avatar generation presents a significant engineering challenge, primarily due to the conflict betw…
Boost CTR Prediction for New Advertisements via Modeling Visual Content
Tan Yu, Zhipeng Jin, Jie Liu +3
Existing advertisements click-through rate (CTR) prediction models are mainly dependent on behavior ID features, which are learned based on the historical user-ad interactions. Nev…
KALAHash: Knowledge-Anchored Low-Resource Adaptation for Deep Hashing
Shu Zhao, Tan Yu, Xiaoshuai Hao +2
Deep hashing has been widely used for large-scale approximate nearest neighbor search due to its storage and search efficiency. However, existing deep hashing methods predominantly…
HIVE: Hidden-Evidence Verification for Hallucination Detection in Diffusion Large Language Models
Guoshenghui Zhao, Tan Yu, Weijie Zhao
Diffusion large language models generate text through multi-step denoising, where hallucination signals may emerge throughout the trajectory rather than only in the final output. E…
FACTS About Building Retrieval Augmented Generation-based Chatbots
Rama Akkiraju, Anbang Xu, Deepak Bora +35
Enterprise chatbots, powered by generative AI, are emerging as key applications to enhance employee productivity. Retrieval Augmented Generation (RAG), Large Language Models (LLMs)…
BOAT: Bilateral Local Attention Vision Transformer
Tan Yu, Gangming Zhao, Ping Li +1
Vision Transformers achieved outstanding performance in many computer vision tasks. Early Vision Transformers such as ViT and DeiT adopt global self-attention, which is computation…
S-MLPv2: Improved Spatial-Shift MLP Architecture for Vision
Tan Yu, Xu Li, Yunfeng Cai +2
Recently, MLP-based vision backbones emerge. MLP-based vision architectures with less inductive bias achieve competitive performance in image recognition compared with CNNs and vis…
In Defense of RAG in the Era of Long-Context Language Models
Tan Yu, Anbang Xu, Rama Akkiraju
Overcoming the limited context limitations in early-generation LLMs, retrieval-augmented generation (RAG) has been a reliable solution for context-based answer generation in the pa…
MoRA: Missing Modality Low-Rank Adaptation for Visual Recognition
Shu Zhao, Nilesh Ahuja, Tan Yu +2
Pre-trained vision language models have shown remarkable performance on visual recognition tasks, but they typically assume the availability of complete multimodal inputs during bo…
Decomposing User-APP Graph into Subgraphs for Effective APP and User Embedding Learning
Tan Yu, Jun Zhi, Yufei Zhang +3
APP-installation information is helpful to describe the user's characteristics. The users with similar APPs installed might share several common interests and behave similarly in s…
MVT: Multi-view Vision Transformer for 3D Object Recognition
Shuo Chen, Tan Yu, Ping Li
Inspired by the great success achieved by CNN in image recognition, view-based methods applied CNNs to model the projected views for 3D object understanding and achieved excellent…
Rethinking Token-Mixing MLP for MLP-based Vision Backbone
Tan Yu, Xu Li, Yunfeng Cai +2
In the past decade, we have witnessed rapid progress in the machine vision backbone. By introducing the inductive bias from the image processing, convolution neural network (CNN) h…