10 papers
WindowQuant: Mixed-Precision KV Cache Quantization based on Window-Level Similarity for VLMs Inference Optimization
Wei Tao, Xiaoyang Qu, Peiqiang Wang +4
Recently, video language models (VLMs) have been applied in various fields. However, the visual token sequence of the VLM is too long, which may cause intolerant inference latency…
Vista: Scene-Aware Optimization for Streaming Video Question Answering under Post-Hoc Queries
Haocheng Lu, Nan Zhang, Wei Tao +4
Streaming video question answering (Streaming Video QA) poses distinct challenges for multimodal large language models (MLLMs), as video frames arrive sequentially and user queries…
Triage: Hierarchical Visual Budgeting for Efficient Video Reasoning in Vision-Language Models
Anmin Wang, Nan Zhang, Wei Tao +4
Vision-Language Models (VLMs) face significant computational challenges in video processing due to massive data redundancy, which creates prohibitively long token sequences. To add…
Attention-Guided Patch-Wise Sparse Adversarial Attacks on Vision-Language-Action Models
Naifu Zhang, Wei Tao, Xi Xiao +5
In recent years, Vision-Language-Action (VLA) models in embodied intelligence have developed rapidly. However, existing adversarial attack methods require costly end-to-end trainin…
MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts
Wei Tao, Haocheng Lu, Xiaoyang Qu +4
One of the primary challenges in optimizing large language models (LLMs) for long-context inference lies in the high memory consumption of the Key-Value (KV) cache. Existing approa…
BAGNet: A Boundary-Aware Graph Attention Network for 3D Point Cloud Semantic Segmentation
Wei Tao, Xiaoyang Qu, Kai Lu +3
Since the point cloud data is inherently irregular and unstructured, point cloud semantic segmentation has always been a challenging task. The graph-based method attempts to model…
MADLLM: Multivariate Anomaly Detection via Pre-trained LLMs
Wei Tao, Xiaoyang Qu, Kai Lu +3
When applying pre-trained large language models (LLMs) to address anomaly detection tasks, the multivariate time series (MTS) modality of anomaly detection does not align with the…
Cocktail: Chunk-Adaptive Mixed-Precision Quantization for Long-Context LLM Inference
Wei Tao, Bin Zhang, Xiaoyang Qu +2
Recently, large language models (LLMs) have been able to handle longer and longer contexts. However, a context that is too long may cause intolerant inference latency and GPU memor…
Value-Driven Mixed-Precision Quantization for Patch-Based Inference on Microcontrollers
Wei Tao, Shenglin He, Kai Lu +5
Deploying neural networks on microcontroller units (MCUs) presents substantial challenges due to their constrained computation and memory resources. Previous researches have explor…
QSpeech: Low-Qubit Quantum Speech Application Toolkit
Zhenhou Hong, Jianzong Wang, Xiaoyang Qu +3
Quantum devices with low qubits are common in the Noisy Intermediate-Scale Quantum (NISQ) era. However, Quantum Neural Network (QNN) running on low-qubit quantum devices would be d…