20 citations · 37 across the 16 of their papers we have counts for
18 papers
Vesta: Queueing-Aware LLM Scheduling for Real-World Workloads
Lancheng Zou, Bowen Li, Wei Mao +6
Agentic applications generate multi-turn sequences of LLM requests, with each new turn extending the context accumulated from prior interactions and tool calls. Prefix caching reus…
D-Quant: Driftable Entropy Coding for KV Cache Quantization
Yi Su, Hong Liu, Guanghua Yu +1
The KV cache has become a major bottleneck in deploying LLMs, as its memory footprint grows linearly with sequence length and batch size, imposing substantial pressure on both memo…
Sample-Guided Exact Top-K Selection for Long-Context Sparse Attention
Siran Liu, Yang Xue, Theo Tang +11
Sparse attention bounds downstream attention work by retaining a fixed-size subset of indexed tokens, but its standalone exact Top- stage must still process materialized score r…
CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention
Yufei Xue, Lin Niu, Hong Liu +6
The quadratic cost of self-attention makes long-context inference prohibitively expensive, and proxy-based block-sparse attention has become a practical remedy. Existing methods ty…
FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling
Xianglong Yan, Hong Liu, Chengzhu Bao +4
Large language models (LLMs) achieve remarkable performance but are expensive to deploy due to their enormous size. FP4 quantization, with formats such as MXFP4 and NVFP4, offers a…
AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding
Hong Liu, Rui Cen, Junhan Shi +10
Speculative decoding accelerates large language model inference without changing the target distribution, but no single drafting structure performs best across real-world workloads…
PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention
Hong Liu, Yuan Cheng, Lin Niu +5
Token-level sparse attention, as implemented by DeepSeek Sparse Attention (DSA) in production systems, makes the downstream attention efficient but shifts the bottleneck to the ind…
D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding
Tianyu Liu, Yuhao Shen, Rui Cen +7
Speculative decoding accelerates large language model (LLM) inference without compromising output quality. Recent parallel drafting methods further improve single-request performan…
DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding
Jiebin Zhang, Zhenghan Yu, Song Liu +9
Block diffusion speculative decoding accelerates LLM inference by predicting all tokens within a block simultaneously for the target model to verify in parallel. Predicting an enti…
Stem: Rethinking Causal Information Flow in Sparse Attention
Lin Niu, Xin Luo, Linchuan Xie +4
The quadratic computational complexity of self-attention remains a fundamental bottleneck for scaling Large Language Models (LLMs) to long contexts, particularly during the pre-fil…
Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought
Tencent Hunyuan Team, Ao Liu, Botong Zhou +248
As Large Language Models (LLMs) rapidly advance, we introduce Hunyuan-TurboS, a novel large hybrid Transformer-Mamba Mixture of Experts (MoE) model. It synergistically combines Mam…
Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation
Zibo Zhao, Zeqiang Lai, Qingxiang Lin +71
We present Hunyuan3D 2.0, an advanced large-scale 3D synthesis system for generating high-resolution textured 3D assets. This system includes two foundation components: a large-sca…
PEARL: Parallel Speculative Decoding with Adaptive Draft Length
Tianyu Liu, Yun Li, Qitan Lv +4
Speculative decoding (SD), where an extra draft model is employed to provide multiple draft tokens first, and then the original target model verifies these tokens in parallel, has…
Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding
Zhimin Li, Jianwei Zhang, Qin Lin +42
We present Hunyuan-DiT, a text-to-image diffusion transformer with fine-grained understanding of both English and Chinese. To construct Hunyuan-DiT, we carefully design the transfo…
EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
Hanlin Tang, Yifu Sun, Decheng Wu +3
Large language models (LLMs) have proven to be very superior to conventional methods in various tasks. However, their expensive computations and high memory requirements are prohib…
E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity
Yun Li, Lin Niu, Xipeng Zhang +3
Traditional pruning methods are known to be challenging to work in Large Language Models (LLMs) for Generative AI because of their unaffordable training process and large computati…
MKQ-BERT: Quantized BERT with 4-bits Weights and Activations
Hanlin Tang, Xipeng Zhang, Kai Liu +2
Recently, pre-trained Transformer based language models, such as BERT, have shown great superiority over the traditional methods in many Natural Language Processing (NLP) tasks. Ho…
TexSmart: A Text Understanding System for Fine-Grained NER and Enhanced Semantic Analysis
Haisong Zhang, Lemao Liu, Haiyun Jiang +14
This technique report introduces TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities. Com…