7 papers
Punctuation-aware Hybrid Trainable Sparse Attention for Large Language Models
Junxiang Qiu, Shuo Wang, Zhengsu Chen +4
Attention serves as the fundamental mechanism for long-context modeling in large language models (LLMs), yet dense attention becomes structurally prohibitive for long sequences due…
Accelerating Controllable Generation via Hybrid-grained Cache
Lin Liu, Huixia Ben, Shuo Wang +4
Controllable generative models have been widely used to improve the realism of synthetic visual content. However, such models must handle control conditions and content generation…
Accelerating Diffusion Transformer via Gradient-Optimized Cache
Junxiang Qiu, Lin Liu, Shuo Wang +3
Feature caching has emerged as an effective strategy to accelerate diffusion transformer (DiT) sampling through temporal feature reuse. It is a challenging problem since (1) Progre…
Accelerating Diffusion Transformer via Error-Optimized Cache
Junxiang Qiu, Shuo Wang, Jinda Lu +4
Diffusion Transformer (DiT) is a crucial method for content generation. However, it needs a lot of time to sample. Many studies have attempted to use caching to reduce the time con…
DiffGAD: A Diffusion-based Unsupervised Graph Anomaly Detector
Jinghan Li, Yuan Gao, Jinda Lu +4
Graph Anomaly Detection (GAD) is crucial for identifying abnormal entities within networks, garnering significant attention across various fields. Traditional unsupervised methods,…
DAMA: Data- and Model-aware Alignment of Multi-modal LLMs
Jinda Lu, Junkang Wu, Jinghan Li +6
Direct Preference Optimization (DPO) has shown effectiveness in aligning multi-modal large language models (MLLM) with human preferences. However, existing methods exhibit an imbal…