13 citations · 31 across the 55 of their papers we have counts for
21 papers · 1 filter
DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement
Renjie Lu, Xulong Zhang, Xiaoyang Qu +2
Unified Multimodal models (UMMs) built on a single architecture have shown impressive performance in both understanding and generation. We identify a fundamental challenge that lie…
WindowQuant: Mixed-Precision KV Cache Quantization based on Window-Level Similarity for VLMs Inference Optimization
Wei Tao, Xiaoyang Qu, Peiqiang Wang +4
Recently, video language models (VLMs) have been applied in various fields. However, the visual token sequence of the VLM is too long, which may cause intolerant inference latency…
From Inheritance to Saturation: Disentangling the Evolution of Visual Redundancy for Architecture-Aware MLLM Inference Acceleration
Jiaqi Shi, Yuechan Li, Xulong Zhang +2
High-resolution Multimodal Large Language Models (MLLMs) face prohibitive computational costs during inference due to the explosion of visual tokens. Existing acceleration strategi…
VLA-InfoEntropy: A Training-Free Vision-Attention Information Entropy Approach for Vision-Language-Action Models Inference Acceleration and Success
Chuhang Liu, Yayun He, Zuheng Kang +2
Vision-Language-Action (VLA) models integrate visual perception, language understanding, and action decision-making for cross-modal semantic alignment, exhibiting broad application…
Vista: Scene-Aware Optimization for Streaming Video Question Answering under Post-Hoc Queries
Haocheng Lu, Nan Zhang, Wei Tao +4
Streaming video question answering (Streaming Video QA) poses distinct challenges for multimodal large language models (MLLMs), as video frames arrive sequentially and user queries…
Triage: Hierarchical Visual Budgeting for Efficient Video Reasoning in Vision-Language Models
Anmin Wang, Nan Zhang, Wei Tao +4
Vision-Language Models (VLMs) face significant computational challenges in video processing due to massive data redundancy, which creates prohibitively long token sequences. To add…
MIRRORTALK: Forging Personalized Avatars Via Disentangled Style and Hierarchical Motion Control
Renjie Lu, Xulong Zhang, Xiaoyang Qu +2
Synthesizing personalized talking faces that uphold and highlight a speaker's unique style while maintaining lip-sync accuracy remains a significant challenge. A primary limitation…
Federated Domain Generalization with Domain-specific Soft Prompts Generation
Jianhan Wu, Xiaoyang Qu, Zhangcheng Huang +1
Prompt learning has become an efficient paradigm for adapting CLIP to downstream tasks. Compared with traditional fine-tuning, prompt learning optimizes a few parameters yet yields…
MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts
Wei Tao, Haocheng Lu, Xiaoyang Qu +4
One of the primary challenges in optimizing large language models (LLMs) for long-context inference lies in the high memory consumption of the Key-Value (KV) cache. Existing approa…
Hierarchical-Task-Aware Multi-modal Mixture of Incremental LoRA Experts for Embodied Continual Learning
Ziqi Jia, Anmin Wang, Xiaoyang Qu +2
Previous continual learning setups for embodied intelligence focused on executing low-level actions based on human commands, neglecting the ability to learn high-level planning and…
RATE-Nav: Region-Aware Termination Enhancement for Zero-shot Object Navigation with Vision-Language Models
Junjie Li, Nan Zhang, Xiaoyang Qu +4
Object Navigation (ObjectNav) is a fundamental task in embodied artificial intelligence. Although significant progress has been made in semantic map construction and target directi…
BAGNet: A Boundary-Aware Graph Attention Network for 3D Point Cloud Semantic Segmentation
Wei Tao, Xiaoyang Qu, Kai Lu +3
Since the point cloud data is inherently irregular and unstructured, point cloud semantic segmentation has always been a challenging task. The graph-based method attempts to model…
VisTa: Visual-contextual and Text-augmented Zero-shot Object-level OOD Detection
Bin Zhang, Xiaoyang Qu, Guokuan Li +2
As object detectors are increasingly deployed as black-box cloud services or pre-trained models with restricted access to the original training data, the challenge of zero-shot obj…
RUNA: Object-level Out-of-Distribution Detection via Regional Uncertainty Alignment of Multimodal Representations
Bin Zhang, Jinggang Chen, Xiaoyang Qu +5
Enabling object detectors to recognize out-of-distribution (OOD) objects is vital for building reliable systems. A primary obstacle stems from the fact that models frequently do no…
Enhancing Multi-Agent Systems via Reinforcement Learning with LLM-based Planner and Graph-based Policy
Ziqi Jia, Junjie Li, Xiaoyang Qu +1
Multi-agent systems (MAS) have shown great potential in executing complex tasks, but coordination and safety remain significant challenges. Multi-Agent Reinforcement Learning (MARL…
ESARM: 3D Emotional Speech-to-Animation via Reward Model from Automatically-Ranked Demonstrations
Xulong Zhang, Xiaoyang Qu, Haoxiang Shi +2
This paper proposes a novel 3D speech-to-animation (STA) generation framework designed to address the shortcomings of existing models in producing diverse and emotionally resonant…
PRENet: A Plane-Fit Redundancy Encoding Point Cloud Sequence Network for Real-Time 3D Action Recognition
Shenglin He, Xiaoyang Qu, Jiguang Wan +3
Recognizing human actions from point cloud sequence has attracted tremendous attention from both academia and industry due to its wide applications. However, most previous studies…
Value-Driven Mixed-Precision Quantization for Patch-Based Inference on Microcontrollers
Wei Tao, Shenglin He, Kai Lu +5
Deploying neural networks on microcontroller units (MCUs) presents substantial challenges due to their constrained computation and memory resources. Previous researches have explor…
EdgeMA: Model Adaptation System for Real-Time Video Analytics on Edge Devices
Liang Wang, Nan Zhang, Xiaoyang Qu +6
Real-time video analytics on edge devices for changing scenes remains a difficult task. As edge devices are usually resource-constrained, edge deep neural networks (DNNs) have fewe…
Shoggoth: Towards Efficient Edge-Cloud Collaborative Real-Time Video Inference via Adaptive Online Learning
Liang Wang, Kai Lu, Nan Zhang +5
This paper proposes Shoggoth, an efficient edge-cloud collaborative architecture, for boosting inference performance on real-time video of changing scenes. Shoggoth uses online kno…
Pose Guided Human Image Synthesis with Partially Decoupled GAN
Jianhan Wu, Jianzong Wang, Shijing Si +2
Pose Guided Human Image Synthesis (PGHIS) is a challenging task of transforming a human image from the reference pose to a target pose while preserving its style. Most existing met…